Ken was an outstanding athlete at Merrill High School where he participated in football, basketball, baseball, and track, and graduated in 1976. He earned ten varsity letters during his athletic career at Merrill High School. Ken was named All-Valley, All-County, and All-State in football during his senior season. He earned All-Central Michigan League accolades in both baseball and basketball. Tolfree still holds the school’s all-time stolen base record and set a Central Michigan League record in 1976 with twenty-five stolen bases in twenty-six attempts. Ken was also a member of the Vandals’ record-setting and state-qualifying relay team in track.
Ken attended Saginaw Valley State from 1976-1979 as a hard-hitting defensive back. He was influenced by Ron Main, his coach at Merrill and his desire to stay close to home to choose Saginaw Valley State College. Saginaw Valley’s football program was in its infancy, so he found early playing time. Ken excelled at reading the quarterback’s eyes and being in the spot that they threw it which led Tolfree to be the current career interceptions leader with twenty-two (22). As a freshman in 1976, he recorded a three-interception game against Adrian. As a sophomore, he had another three-interception game and took one of them back for a 45-yard touchdown against St. Norbert on Saturday October 29, 1977. In that game, he also blocked two punts. For his efforts, he was named the NAIA District 23 Player of the Week. The two-time defensive captain was named to the All-GLIAC teams in 1978 and 1979. Ken led the Cardinals to the NAIA playoffs in his senior year. The game was played in South Carolina but unfortunately, they lost to Presbyterian in front of 4,000 fans. Tolfree was the first SVSU Cardinal to make NAIA First Team All-American and only the third from Saginaw Valley to be named to the list. Twice he won the Saginaw Valley Outstanding Defensive Back award.
After college, he began his career and started a family. Ken and his three brothers, Art, Rick, and Joe teamed up to play in a Saginaw touch football league. One season, they finished second in the nation for their efforts.