Smith was born in Lancaster, Pennsylvania in 1757. During the American Revolution, he served in the Continental Army. After the war, he attended Princeton University and went on to join the Maryland bar. While practicing law, he served as a presidential elector, a state senator, a member of the Maryland House of Delegates, and a member of the Baltimore city council.

Thomas Jefferson appointed Smith as secretary of the Navy in 1801. He intended to resign that post in 1805, when he was appointed as attorney general, but was unable to do so when his proposed replacement was unable to accept an appointment as secretary of the Navy due to poor health. Impressively, Smith served as both cabinet officials simultaneously until John Breckinridge became attorney general later in 1805. Smith returned to serving as secretary of the Navy, a position he held for the rest of the Jefferson administration. President Madison appointed Smith secretary of state in 1809 and thus overseer of the 1810 census.

Smith did not last long as secretary of state, however. He and Madison had fundamentally different ideas of foreign policy. In fact, each man went so far as to publish a written attack against the other before Madison forced Smith to resign in 1811. After his short tour at the State Department, Smith became the president of the American Bible Society. In 1818, he became the founding president of the Maryland Agriculture Society. Robert Smith died in Baltimore on November 26, 1842.