The Gem is on historic Van Vorst Park, an upscale neighborhood of Jersey City. In 1851, Van Vorst was annexed by Jersey City. The neighborhood contains nineteenth century rowhouses and brownstones. Landmarks include Jersey City Museum, Barrow Mansion, Dixon Mills, Jersey City Public Library, Jersey City City Hall and the Heppenheimer Mansion. The Grove Street PATH station is located nearby to the north and is the Jersey Avenue to the south.