During planning stages the construction death toll was estimated to be one worker per floor, or over 100 workers overall. However, only a handful of workers lost their lives during construction. i.e just about 5 people died. Following is a pic of the men involved in construction of Empire State Building. In this picture they are having lunch without any security measures, on a steel beam high in the air.