This time it's not a recording
Neil Diamond will at last sing 'Sweet Caroline' at Fenway this week. Not to mention songs from his - surprise - recent No. 1 album.
There's something almost mystical about the prospect of Neil Diamond singing "Sweet Caroline" in the flesh at Fenway Park next Saturday. The song, which blasts from the speakers at every home game, has become the team's unofficial anthem and more: a rousing eighth-inning singalong-turned-musical totem whose propitious properties remain a mystery - even to the songwriter who the folks at Fenway have been trying to lure to the park for years.
"I'm not sure I understand it, but it's fun to sing along with and for me it's always been a good-luck song," Diamond says of his 1969 hit, whose popularity has hardly ebbed in four decades. "It's been around for so long, and I think that anybody who adopts it has some of that good luck. The fact that the Red Sox have adopted it proves the point."
Uh, really?
"I put home runs and good pitching in front of the song," he concedes. "But the song does capture the spirit of the team."
Diamond, meanwhile, has been chasing down spirit of a different sort. His last two albums are stripped-down, singer-songwriter affairs, rich with stocktaking and largely devoid of the saccharine and glitz the artist deployed for decades. Masterminded by Rick Rubin, the producer responsible for Johnny Cash's similarly spare, late-career comeback records, 2005's "12 Songs" and this year's "Home Before Dark" have touched a chord with fans and critics (including this one) who've had little stomach for Diamond's histrionics.
Indeed, "Home Before Dark" is Diamond's first album to reach No. 1 on the US charts, an accomplishment that took the artist by surprise.
"I thought 'Hot August Night' was a No. 1 album, but maybe not in Billboard," Diamond says of his 1972 live album. "It hasn't hurt my career."
Diamond's career has been enormously successful by conventional standards. He's sold more than 120 million albums worldwide and maintained a large, devoted following for his live shows. But his image as an artist is divided, and for good reason: Diamond is the king of kitsch and also a songwriter's songwriter. Watching "The Last Waltz" recently, I was startled to see Diamond onstage alongside folk-rock icons like Joni Mitchell and Neil Young. (The Band's Robbie Robertson had recently produced Diamond's concept album "Beautiful Noise.")
Most musicians choose to be one or the other: sequined showman or contemplative tunesmith. Diamond embraces both.
"People are complicated. There are many facets and I like to explore as many of them as possible," he says. "I think writing a song like 'America' [a bombastic interpretation of the immigrant experience] is as valid as writing a song like [the comparatively downbeat confessional] 'I Am . . . I Said.' That's why they're still around and that's why I still do them in concert."
Diamond was baffled when Rubin, whose resumé was filled with rap and rock projects from the likes of Public Enemy, the Beastie Boys, and Red Hot Chili Peppers, approached him about collaborating. But the artist was intrigued when Rubin floated a novel idea: Why didn't Diamond take a close listen to his earliest work, introspective records that pre-dated the symphonic pomp and circumstance of his big hits? Diamond did - and realized that he had something to say that hadn't been coming through.
"Rick was able to hone in on the simplicity of the songs I'm writing and not make a big deal about the production elements. It does put a terrible onus on the songs," Diamond says. "There's no dressing and frosting and bells and whistles. So it has to have substance to work. The songs are still about relationships, because that's what I know least about, so I guess I write about it to try to understand it. But there is a maturity and perspective that comes with age. Sometimes it's painfully gained, but as far as songs are concerned it gives depth and color that you can't get anywhere else."
More than 40 years into his career, when many of Diamond's peers are coasting on past accomplishments and recycling the back catalog, songwriting remains the center of his universe.
"I was blown away by how committed and how open-minded he was," says Smokey Hormel, a session guitarist who has worked on many of Rubin's projects, including Johnny Cash, the Dixie Chicks, Justin Timberlake, and the past two Diamond albums. "We would get to the studio and Neil would already be there, and we'd work all day and in between takes he'd be adjusting a word here or there to make it perfect, and we would leave and he was still there. Neil would try anything if he thought it would make the song better. He was completely fearless and not at all self-conscious."
Contrary to common wisdom, time and experience haven't brought Diamond any measure of ease or comfort with the creative process. He says he doesn't feel that he has to keep proving himself, but rather that each album feels as vitally important as the first one. He's quite simply driven, and at 67, outside of visiting with his grown children and reading an occasional good book, Diamond has little interest in anything besides music. He still loves touring because it's a respite from the grueling, 24/7 process of writing. His to-do list is short.
"When I was a teenager, I wanted to be a research biologist and discover the cure for cancer," says Diamond. Pressed for a more realistic game plan, he says, "I tell you, I'd like to write as many wonderful songs as I possibly can before I disappear from this planet."
Joan Anderman can be reached at anderman@globe.com. For more on music, go to www.boston.com/ae/music/blog. 