The 1930s were a tough time. While the economy recovered, the decline of the Armory-Gymnasium continued for 60 years.
Few alterations or improvements were made during the 1930s and 1940s. The only repair recorded for this period was 21 squares of Barrett Black Diamond pitch and felt roofing that were installed in October 1940, probably on the "head house," over the south entrance.
By the early 1950s, the annex and the gymnasium were considered antiquated, and planning for demolition began. The Wisconsin Alumni magazine announced in 1951 that "according to a plan recently initiated by Paul J. Fisher, `22, the cramped Armory pool will be the first of the building's inadequate facilities to be replaced." After the annex was razed in 1956, athletic activities that had been held in the annex were housed in the Camp Randall Memorial Sports Center.
Demolition of the gymnasium itself was planned for some time after 1953. Alexius Baas, a columnist for the Madison Capital Times, lamented that it was "Too bad the old gym must be torn down. We could well imitate other universities in this country and abroad and preserve our historic buildings with their memories and traditions."
Campus leaders in 1956 called the physical education facilities "woefully inadequate and outmoded" and "easily the worst in the Big Ten." In 1957, the Board of Regents asked the Campus Planning Commission to investigate a possible site for a new gym, which would include a new pool and space for fencing, gymnastics, and other sports, for both physical education and intercollegiate athletics. After considerable discussion, the Regents selected a site in September 1958, and approved plans in 1960, but it was not until the fall of 1963 that the gymnasium, Unit I, and natatorium--located on Observatory Drive near the intramural playing fields--was ready for student use. It had Olympic-size swimming and diving pools, which were considered by President Conrad A. Elvehjem to be a great improvement over "that bath tub in the old red gym."¬†It was anticipated that the old gymnasium would be demolished soon after the opening of this new building. In 1965, demolition was postponed until after another gymnasium, called Unit II, was added to the natatorium on the west end of the campus (opened in 1967). With the construction of these new facilities, the original armory and gymnasium was regularly termed the "old red gym."
The gymnasium continued to be used heavily during the 1960s as an athletic facility and headquarters for ROTC. A proposal was made to remodel the north end of the pool room with men's showers in 1969. Otherwise, few repairs were made during this period, when its demolition apparently loomed only a year or two in the future.
The Red Gym became front-page news after arsonists firebombed the building early on the morning of January 2, 1970. Flames caused extensive damage to the southeast tower used by students in the Water Resources Management Program, not by the ROTC, the arsonist's target. It took more than seven hours to bring the blaze under control; the Capital Times reported that new fires kept breaking out, "fanned by drafts and carried through the walls and heating ducts." There was extensive damage to a locker room. The ROTC offices in the southwest tower were not damaged.
Since its construction, the armory and gymnasium facilities had been intended for the use of men. In November 1971, however, about 15 women began playing basketball on an upstairs court and refused to follow instructions from an attendant, who told them to leave and that they were "carrying this women's lib thing too far." The women gained permission from the director of men's recreational facilities to continue playing and scheduled a meeting for the next week to demand the right to use the showers. It was not until 1973, as part of a program "to achieve a greater degree of equality for women staff and students in athletic programs and facilities for the coming year" that the university authorized expenditures for "separate locker room, shower and toilet facilities for the Unit II gymnasium, Lathrop Hall, and the red gym." The project included the construction of a partition in the shower and locker rooms. A delay in making the changes prompted more protests by women, who renamed the area around the pool a "People's Locker Room" and dressed in the same room with the men. In 1974, 108 lockers along the west side of the first floor and ten showers were set aside for women, partitioned off by green vinyl curtains.
Also in 1974, after some controversy, the Red Gym was included in the Bascom Hill Historic District and listed on the National Register of Historic Places, as a part of the university's 125th anniversary celebration.
Alums remember the Red Gym as the site for student registration. When registration was moved elsewhere in the fall of 1983, it ended the last large, all-campus function for the gym.
Over a period of 60 years, the historic structure had become a woefully underused building at the crossroads of a very busy central campus complex.