Japan has been severely impacted by natural disasters including but not limited to earthquakes such as the Great Hanshin-Awaji Earthquake in 1995 and the Kumamoto Earthquake in 2016. These seismic events have underscored the high number of casualties that result from individuals becoming trapped in collapsed buildings or affected by fires, thereby accentuating the need for building-specific earthquake assessments. Although experts have performed detailed analyses using automated satellite imagery and UAV-captured photos for longer-term objectives such as secondary disaster prevention, reconstruction, and insurance claim verification, these require a substantial amount of time to implement. This paper introduces two methods that employ UAVs to rapidly detect anomalous building structures, allowing for the simultaneous observation of multiple buildings. First, we present a method for identifying building structures from incomplete three-dimensional point clouds acquired by unmanned aerial vehicles (UAVs) that move faster over the target area in a limited time for rapid assessment. The method efficiently identifies the structural characteristics of buildings by operating under certain geometric assumptions such as the angles between building sides being approximately 90 degrees and vertical consistency in building shape. We also present a method for identifying collapsed buildings by extracting features from point clouds in Fisher vector and normal histogram and using a machine-learning model for detection. In our evaluations, we have shown that by limiting observations to less than half of the building structure, the first method can successfully recognize the geometric shape of 70% of the undamaged buildings. In the experiments for the second method, both feature extraction methods achieved a Receiver Operating Characteristic Area Under the Curve (ROC-AUC) values greater than 0.99.