After graduating from Brooklyn College with a degree in English and serving in the United States Army, Moss began his music career by promoting "16 Candles", a hit for the Crests on Coed Records. In 1960 he moved to California, where he teamed up with Alpert, forming Carnival Records in 1962 and running the company from an office in Alpert's garage. Discovering that the name was already taken, they dubbed their new-found company A&M Records.
Moss and Herb Alpert were inducted into the Rock and Roll Hall of Fame in 2006 as non-performers.
Jerry Moss, is a longtime horse-breeder and owner who received the largest ever first-place purse from the Kentucky Derby in 2005 after the victory of the first horse he had ever entered in that race, Giacomo.
In 2011, he was inducted into the Southern California Jewish Sports Hall of Fame.