Your body isn't feeling temperature it is feeling the amount of heat leaving your body. There is a subtle distinction here. You have 3 variables, the amount of heat the metal can pull away from your body (heat capacity). The amount of heat the metal can store (heat capacity * density, keep in mind metals are heavy). The speed at which the metal can pull away this heat. (thermal conductivity). A material which does not conduct heat well will feel warm, because your hand heats up the surface and the heat stays at the surface. This is why plastics feel warmer then metal.
Metal has a lot of density, a lot of capacity to store heat, and a very good conductivity meaning it can pull the heat out of your body and into the metal very well and it can store a lot of heat. This is also why different metals feel different temperatures at room temperature.
Feel free to write