Gigantism does NOT require every doorway to be enormous. Use recognisable architectural components — arches, galleries, columns, aqueducts, bridges, stairs, chambers — then repeat them at incomprehensible scale. The player should understand individual pieces while failing to fully comprehend the larger structure. Human-scale combat can then remain readable without weakening the Tower.
THE PEOPLE DO NOT NEED TO BECOME MICROSCOPIC FOR THE TOWER TO FEEL IMMENSE.