The building under the San Francisco side of the Golden Gate Bridge is Fort Point, built by the US Army at the height of the Gold Rush between 1853 and 1861 as part of a defense system to protect the harbor from foreign attack.
When it was no longer needed as a defense position, it served as barracks and military accommodation, but was eventually abandoned after World War Two and fell into disrepair.
While Fort Point never saw battle, it predates the American Civil War and construction of the Golden Gate Bridge, and is recognized for its architectural and historical significance. In 1970, it became a National Historic Site and is now managed by the National Parks Service and open to the public. Entry is free and self-guided tours are available.