In one of the most memorable moments of the 1992 presidential campaign, Arkansas governor Bill Clinton appeared on The Arsenio Hall Show and played Elvis Presley’s “Heartbreak Hotel” on the saxophone, along with the show’s house band. Here’s a look at some other presidents (and presidential candidates) who showed off their musical talents.

Harry Truman

Truman started taking piano lessons from his mother when he was 10 years and old, and he must have gotten pretty good—he would wake up at 5 a.m. every morning for two hours of practice, a habit he kept up well into adulthood and into his presidency. Truman made an appearance at a county fair in Missouri in 1945, and sat down at the piano and performed. Afterward, he remarked, “When I played this, Stalin signed the Postdam Agreement.” He’d also entertain visiting White House guests and reporters with his prowess on the keys.

Ross Perot

Campaigning as an alternative to the two major party candidates, the Texas billionaire ran as a third-party candidate in both 1992 and 1996. Also a little bit different was Perot’s choice of instrument: He plays the accordion. (Richard Nixon also played the accordion, but when he played an instrument in public, it was usually the piano.)

Mike Huckabee

The former Arkansas governor and two-time presidential candidate took up the electric guitar after seeing the Beatles on The Ed Sullivan Show as a teenager in 1964, eventually switching to bass. Since 1996, he’s played the instrument in a rock n’ roll cover band called Capitol Offense. They often played at Huckabee campaign events.

Martin O’Malley

The Maryland governor briefly campaigned for the 2016 Democratic nomination. While making the media rounds, O’Malley put in an appearance on The View, where he sang and played guitar on a performance of Taylor Swift’s hit “Bad Blood.” Rolling Stone called it “a pop culture war crime,” and O’Malley more or less agreed. “If only I’d done it in the key I practiced, it’d have been a little bit better.”