Harry Davis was born in Cleveland, the son of Jacob Henry and Rosalie Dete Davis. He attended Hiram College briefly (1904-1905) before transferring to Western Reserve University. He graduated from its law school in 1908. Davis devoted his entire adult life to public service in his native city. He was a counselor, attorney, Masonic officer, and early member and proponent of the National Association for the Advancement of Colored People and the Association for the Study of Negro Life and History. He was an ardent worker for his church and a passionate believer in American democracy.
Davis began his career in politics in 1921, when he was elected to the Ohio Legislature. Eventually, he served four consecutive terms in the Ohio House, and was a member of the Cities Committee and the Codes and Taxation Committee. Throughout his terms, he was alert for potentially anti-African American legislation. Davis is best remembered for introducing House Joint Resolution No. 5, a proposal to amend the Ohio Constitution by striking the words "White Male" in the section regarding voting rights. Ohio voters adopted the amendment in 1923.
In 1928, he became the first African American member of the Cleveland Civil Service Commission, on which he served until 1934, when he was elected to the Cuyahoga County Charter Commission. In 1947, he returned to the Ohio Legislature as State Senator, a position no African American had filled in fifty-four years. As a senator, he helped block passage of two discriminatory pieces of legislation, an anti-interracial marriage bill and a bill calling for segregating Ohio's schools. His constituents were appreciative. On September 6, 1947, the Cleveland Call & Post characterized Davis as "the best legislator in the state" and "the most able member in a quarter century."