First try: the LED lit up no matter the pushbutton was pressed or not.
Second try: changed the position of the wire connecting to the switch and it worked!
When switch is on: switch – 0V; resistor – 0.65V; LED – 2.65V
When switch is off: switch – 2.10V; resistor – 0V; LED – 0V
The first time both of the LEDs didn’t light up. With helps from two angels: Viola and David, I understood that the Arduino Nano only provides 3.3V which is not sufficient for two LEDs.
So I changed the position of the input wire and pinned it into the “VIN”, and it worked. Now the totally voltage of this circuit is 5V from the USB connector.