James Edward Taylor was born in Florence, Alabama, on January 9, 1950. He is the oldest son of Robert and Margaret Taylor of Florence. His father is a retired high-school physics and chemistry teacher; his mother is a retired high-school English and typing teacher. His younger brother, Jack, lives and works in Birmingham, Alabama.

Jim Taylor began ice skating when he was ten years old and was hooked instantly. His parents encouraged his interest, and he won several regional and national competitions, including a gold medal at the 1974 U.S. National Figure Skating Championships. Subsequent injuries prevented him from making the U.S. Olympic Team, but he did serve as an advisor to the team that competed in the 1980 Winter Olympics in Lake Placid, New York. During this time, Jim also made a considerable personal fortune by appearing in television and print ads for a variety of products including breakfast cereal and shaving cream.

Through a mutual friend he met Sandy Ross, a former ballerina and dancer. Sandy and Jim were married on June 5, 1974, in Florence, Alabama. They then moved to Oxford, Mississippi, where Sandy took a job in the dance and theatre department of The University of Mississippi. Jim worked from home as a venture capitalist, adding to his family fortune through a number of shrewd real estate transactions. Jim and Sandy had their first child, a boy, Shane, on April 4, 1975. Their second child, a boy, Dale, was born on February 9, 1978. Their third child, Crystal, a girl, was born on January 28, 1979.

Although Shane showed no interest in skating, Dale and Crystal took an immediate liking to it, and Jim became their first mentor and coach. Dale and Crystal's skating successes accumulated over the years, and Jim was certain that his children would make the shortlist for the 1998 Nagano Olympics. But after a series of disappointing finishes in the 1997 championship season, the pair failed to make the team.

Dale and Crystal Taylor were reported missing in January of 1998 after they failed to show up for a training camp in Colorado. Their bodies were found in a freshly-dug grave at St. Peter's Cemetery in Oxford two weeks later. While all the Taylor family members came under suspicion during the investigation, detectives finally concluded that Dale killed Crystal and then was murdered himself by a family friend, Christian Chambers.

The investigation also brought to light evidence that Crystal had been sexually abused at an early age. In April of 1998, Jim Taylor was arrested on charges of "touching a child for lustful purposes" and pleaded nolo contendere. The charge was reduced through plea bargaining because of the minimal amount of evidence, and Jim was sentenced to 38 months imprisonment, a $5,000 fine, 5 years probation, and counseling. He was also required to register as a sex offender. He served his full sentence from July 1998 to September 2001, when he returned to Oxford.

Since his release, Jim has been living in a rented apartment and doing clerical work at Monroe Real Estate. Keith Monroe, a long-time friend of Jim's going back to Jim's days as a real estate entrepreneur, gave him the position to help him get back on his feet. Many of Jim's other friends have distanced themselves from him, and Jim has not returned to playing most of the sports he so enjoyed before his arrest.

The loss of Dale and Crystal and the ensuing investigation strained relations between Jim and Sandy. Sandy escaped charges in the case after receiving a grant of immunity in exchange for testimony against her husband. While Jim was in prison, Sandy filed for divorce, to which Jim consented. Sandy moved to New Orleans in 1999, where she works at her alma mater, Tulane University, in the Department of Theatre and Dance.

Jim and Sandy's surviving son, Shane, was also traumatized by the Dale's and Crystal's deaths. He dropped out of the University of Mississippi, where he'd been working part-time toward a music degree, and moved to Seattle, Washington, where he worked at Café Flora during the day and played with a grunge band, The Mop Heads, by night. In 2000 he enrolled at the University of Washington intending to major in economics. Now finishing his sophomore year, Shane plans to obtain an MBA and seek a career in the music recording industry. Shane has visited his mother in New Orleans on several occasions, but refuses to see Jim.