Growing up in New Orleans in the 1950s, Art Neville drank deep at the wellspring of American music. Rhumbas and boogies, mambos and waltzes, ragtime and blues, calypso and jazz—it was all there, a musical gumbo rising unsummoned out of the streets. Turn down one block and you might hear something like the call and response of a black Baptist church. Turn down the other and you might hear something from the Caribbean. A Spanish inflection here, a French one there. Mostly what Art remembers though, were the second-line rhythms of the street parades.

A dance tradition native to New Orleans, second-line dates to the early 19th century, when jazz bands would march in the street to commemorate the dead. Once the body was buried, the band (the first line) would break out into a raucous music, and the crowd (the second line) would follow them into the street, dancing as they went. The tradition spread to all parades in the city, culminating with Mardi Gras, when the entire city showed up to second line.

But the entire city wasn’t equally welcome. Despite its roots in black cultural forms, Mardi Gras parades were designed for whites. As a child, Art remembers running through white neighborhoods, where floats threw the bulk of their parade loot, bending to scoop up the rubber cigars, whistles, beads and necklaces, while trying to avoid the police. “Mardi Gras might look like a time when all hell breaks loose and boundaries are busted,” Art says in The Brothers Neville, the autobiographical oral history of one of New Orleans’ most important musical families.“But in the Big Easy boundaries are always there.”

Art’s only escape from those boundary divisions was in the music. As a kid, messing around on a piano in his aunt’s house, Art let instinct, rather than genre, be his guide. People told him he was playing barrelhouse, a term that associated more with the fermented stink of beer at a local piano bar than any form of music. Art, like many of the musical savants coming out of the Crescent City, never learned to read music; he picked it up in piano bars, watching and listening to guys like Professor Longhair and his uncle Jolly, a man with Native American ancestry, who was heavily involved with the local Mardi Gras Indian chapter. When Art sat down at the piano, the music came to him naturally, without a struggle. “Didn’t think about styles or scales. My fingers worked the keys like they had a mind of their own.”

Art attended a local Catholic high school with Allen Toussaint, who brought him into the orbit of Cosimo’s Recording Studios, the birthplace of New Orleans rock ‘n’ roll. There, in a tiny studio in the French Quarter, Little Richard recorded a number of his ‘50s hits, including “Tutti Frutti” and “Girls Can’t Help It”—a tune that features Art singing backup. Before Art had graduated from high school, he’d put together a local group, the Hawketts, scored a local hit, “Mardi Gras Mambo,” and opened for Ray Charles.

This was ‘50s New Orleans: geniuses were lurking around every corner. Fats Domino had sold a million copies of his hit “The Fat Man”—the first rock ‘n’ roll single to break that mark. Professor Longhair, whose rhumba-inflected piano blues were too weird, too original to sell records, was playing in neighborhood bars nightly. (“Fess,” Art says, “was a fountainhead of New Orleans music.”) Cousin Joe, who would later go on to record with clarinetist Sidney Bechet, was busking in the French Quarter, playing in a distinctly New Orleans style that, to Art, felt older than rock ‘n’ roll, rhythm and blues, or even ragtime or jazz. “Whatever it was,” Art says, “it was right.”

While New Orleans was a hotbed of musical talent, it was also a battleground in the South’s reactionary war against the Civil Rights movement. New Orleans, with its long history of racial ambiguity, resorted to even more pernicious and bizarre means of enforcing discrimination. Some clubs used a brown paper bag to determine which skin tones could enter the club; others hung a fine-tooth comb over the doorway to deny entrance by the thickness of one’s hair.

Brothers is as much a history of American racism as it is American music, particularly the way New Orleans police declared war on the black neighborhoods, where the Neville brothers grew up. By the mid-‘60s, all four of the Neville brothers had done time in the local parish prison for crimes as innocuous as possessing a joint or two. To be young and black in New Orleans was to be constantly harassed by the cops. Art’s younger brother Charles ended up doing two years in Louisiana’s Angola prison, notorious among prisons for inhumane conditions—a place that had more in common with an antebellum plantation than a correctional facility.

Things weren’t much better outside the prison walls. In the early ‘50s, when Charles Neville went on tour with local singer Gene Franklin, newspapers refused to print pictures of black artists. Not even a singer whose genius was as nationally recognized as Ray Charles. So while touring the country, Gene, who could do pitch-perfect imitations of both Ray Charles and B.B. King, took to billing himself as either one. Night after night, their band played to sold out crowds who thought they were hearing the genuine thing. When another local singer Larry Williams headed out on tour, he called up Art and his band, the Hawketts, with a similar idea. He wanted to double book a tour. Each night, Art would dawn Larry’s clothes, play his piano licks, sing his songs—sometimes even in the same town. If white America wasn’t ready to put black genius in the spotlight, that was fine by Larry and Art. They’d simply hit them for double. The mostly white audiences never had a clue.

Despite these small victories, by the mid ‘60s, the Neville brothers were worn out and broke, their meager royalties exhausted by corrupt managers. After long stints on the road, in the Army, and in jail, the brothers (with the exception of Charles) finally regrouped in New Orleans, where they began jamming at a local bar called the Nitecap. It was mostly a family affair: Aaron and brother Cyril took turns singing while Art manned the piano, backed by a group of local musicians—drummer Joseph Modeliste (Zig), his cousin, bassist George Porter Jr., and guitarist Leo Nocentelli—all from the same Valence Street neighborhood where the brothers had grown up. Attendance, at least at first, was sparse. Art: “You could’ve thrown a hand grenade in there, and it wouldn’t have killed nobody but the band.”

So when the owner of The Ivanhoe, a club down on Bourbon Street, offered Art a steady gig on the condition he bring no more than three musicians with him (the tiny bandstand couldn’t support any more), he couldn’t say no. He split up the family band, and took the rhythm section with him. Art felt that they needed a new moniker, something that reflected their music, a strange measured approach to funk. The Meters were born.

The Meters didn’t invent funk music. And they certainly weren’t the first group to make a hit record without a singer. But never before had an instrumental band attacked the inherited template for rhythm and blues with such a sophisticated degree of syncopation. To listen to the four of them let rip on a tune like “Sophisticated Cissy” is to hear a drum line put to music. No matter that only one of them is actually playing the drums: They trade off carrying the beat the way jazz musicians trade solos. “Musicians talk about ‘the one,’ the primary beat,” Art says. “Man, I never knew where ‘the one’ was. My sense of syncopation was all screwed up.”

The reformed four-piece was an instant hit in New Orleans, so much so that jealous patrons started leaving bullets inscribed with the band members names in the Ivanhoe’s tip jar. It didn’t take long for Art’s old school pal Allen Toussaint to get wind of what was happening and bring the group over to Cosimo’s, where they became something like the de facto house rhythm section, anchoring legendary recordings by Dr. John, Lee Dorsey, and Betty Harris. For a period from the late ‘60s to the early ‘70s, the Meters became synonymous with the New Orleans sound, the musical analogue to what Booker T and his MG’s were doing over at Stax Records in Memphis.

The music that ended up on their first record, 1969’s The Meters, came out of that early session work. These weren’t so much songs as they were musical asides, impromptu jam sessions caught on tape. “We'd be doing some Lee Dorsey or Betty Harris or something and Toussaint would get tired,” Art says. “He'd split and we'd still be fired up and say, 'Well, hell, we oughta cut a couple of tracks.'"

What stands out about those tracks—and to some degree, the two records they recorded afterward, Look-Ka Py Py and Struttin’—are the silences. They still sound explosive now, a half-century after they were recorded. On the first track, “Cissy Strut,” their first single, Art’s organ isn’t even really audible until about halfway through. The three others slink around like a team of boxers, darting in and out, landing deft jabs and hooks on the same punching bag of a beat. On “Sophisticated Cissy,” the other single off The Meters, Art glides over the beat without ever getting in the way. “Early on, I learned to lay back and let the singer or guitarist or saxophonist lead,” Art says. “I’d never play over him. I’d play around him. A note here, a lick there. I’d come at it from an angle.” Listening closely, this music is all angles, the spaces between notes. It’s a challenge, trying to find a moment where they all touch down together. You’d need better audio software than what I have on my phone—the equivalent to Muybridge’s stop-motion photography—to capture it: the horse with all four feet off the ground.

There’s not much in the way of singing on those early recordings. When vocals do make their way into the mix—like on “Look-Ka Py Py”—its usually in the form of syncopated grunts, an early kind of beat-boxing. “We shouted out some stuff—couldn’t even call ‘em lyrics—that you’d hear on the streets of New Orleans,” Art says. “God knows what it meant.”

That changed in 1972, when the band’s contract with Josie Records expired. Their label had gone bankrupt. Warner Bros offered them a new contract, but with conditions. Namely, they wanted Allen Toussaint in the studio. While he’d been listed as a producer on their early records, Toussaint had had, in fact, very little to do with what made it to tape. “The only work I had to do with the Meters was open the door,” he once joked. Starting with Cabbage Alley though, Toussaint took strict charge of the band’s new major-label budget. Gone were the jammed-out improvisations, replaced by thoughtful arrangements, soulful lyrics, and a lush, overdubbed sheen. On Cabbage Alley, the Meters cover a sad Neil Young song.

While they lost the smoltering energy of their early recordings, the Meters got more in touch with their roots on Cabbage Alley, pushing musical pins onto the map of the African diaspora they’d been hearing all their lives. They made a trip down to Trinidad and Tobago, where they played with The Mighty Sparrow, “The King of Calypso.” On “Soul Island,” a steel-drum-tinged song off Cabbage Alley, they pay their respects to calypso, with a guitar solo that sounds as it it were ripped note for note from a ‘70s West African high-life record.

Their second attempt for Warner Bros came in 1974 with Rejuvenation. Same sheen and studio polish, but with more of their earlier spunk. The result was some of their most rhythmically complex music to date: "People Say," "It Ain’t No Use," "Loving You is on My Mind.” Their continued interest in the roots of their music was reflected in tunes like “Africa.” Most notably though, for the first time, the band embraced the second-line rhythms of their New Orleans childhood on “Hey Pocky A-Way.”

To understand what’s happening in “Hey Pocky A-Way” requires some understanding of what was happening in New Orleans in the mid-18th century. A good century before the jazz funeral emerged, when New Orleans was still a French colony, African slaves, Native Americans, and racially-mixed free people of color began meeting regularly at the Place des Negrès, an open area behind the city that had once been a sacred site of Houma Indian corn feasts. Communal trading and recreation sprung up between the marginalized groups. Religious and musical traditions began to blend together. These weekly gatherings grew to encompass drum ceremonies, the elaborate communal rituals through which enslaved Africans, against all odds, maintained a link to their ancestral past and familial lineages that had been destroyed by slavery. Here, in an area that would come to be known as Congo Square, they danced and sang, traded merchandise, and engaged in cultural exchange of a dizzying kind, on a scale unrivaled in human history.

These are the rhythms we hear in “Hey Pocky A-Way,” the same ones that Jelly Roll Morton recounts hearing as a child, some 70 years prior to Rejuvenation, the same rhythms that Art and his brother Charles used to tap out on cigar boxes after the street bands passed their window. Art worked up the piano part as an homage to those early memories of syncopation. “I tried to make it original—and I think it was—but I also believe there is no originality,” Art says. “All we can do is put together old pieces in new ways.”

There’s a whole tradition of these Mardi Gras Indian chants breaking through into popular culture. Sugar Boy Crawford and the Cane Cutters cut “Jock-A-Mo” in the early ‘50s, which was made slightly more famous as “Iko Iko” by the Dixie Cups, who sang with Art’s sister Athelgra. “‘Iko Iko’ is another one of those big easy anthems,” Art says. “A Mardi Gras Indian chant I can’t explain, don’t want to explain, but it’s something I’m still singing fifty years later.” You hear that same style of Mardi Gras chant in the chorus to “Lady Marmalade” (that infectious itchi – gitchi – ya-ya – da-da bit), a track the Meters initially recorded with Patti Labelle in 1974, which went on to have a second life via Christina Aguilera and the Moulin Rouge soundtrack in 2001. It’s still here, is all I’m saying. It’s all around us. Old pieces coming together in new ways.

—Ryan Marr