Born in Islington, London, the son of a naval petty officer who organised concert parties, Connor first appeared on the stage at the age of two as an organ-grinder's monkey in one of his father's shows, in Portsmouth. By 11 years old, he had his own act. He attended the Central School of Speech and Drama, where he was a Gold Medal winner. Connor made his professional debut in J. M. Barrie's The Boy David, at His Majesty's Theatre, London in December 1936. During World War II he served as an infantry gunner with the Middlesex Regiment but continued acting by touring Italy and the Middle East with the Stars in Battledress concert party and ENSA. While waiting to be demobbed in Cairo, Connor received a telegram from William Devlin asking him to join the newly formed Bristol Old Vic, where he gained a solid grounding in the classics. He moved on to the London Old Vic Company for a 1947–48 season at the New Theatre. His most notable performances there were as Chaplain de Stogumber in Saint Joan and Dobchinsky in The Government Inspector, which starred Alec Guinness. Realising he was not a tall, impressive juvenile lead or a young lover type, he decided to specialise in comedy.