Slaves in the Family Read online

  “One day you’ll want to know about all this,” Dad said, waving his hand vaguely, his lips pursed. “Your ancestors.” The tone of the old joke was replaced by some nervousness.

  I know my father was proud of his heritage but at the same time, I suspect, had questions about it. The story of his slave-owning family, part of the weave of his childhood, was a mystery he could only partly decipher. With the gift of the book, Dad seemed to be saying that the plantations were a piece of unfinished business. In that moment, the story of the Ball clan was locked in the depths of my mind, to be pried loose one day.

  When I was a child, Dad used to tell stories about our ancestors, the rice planters. I got a personal glimpse of the American Revolution, because the Balls had played a role in it—some of us fought for the British, some for independence. The Civil War seemed more real since Dad’s grandfather and three great-uncles fought for the Confederacy. From time to time in his stories, Dad mentioned the people our family used to own. They were usually just “the slaves,” sometimes “the Ball slaves,” a puff of black smoke on the wrinkled horizon of the past. Dad evidently didn’t know much about them, and I imagine he didn’t want to know.

  “Did I ever tell you about Wambaw Elias Ball?” he might say. “His plantation was on Wambaw Creek. He had about a hundred and fifty slaves, and he was a mean fella.”

  My father had a voice honed by cigarettes, an antique Charleston accent, and I liked to hear him use the old names.

  “Wambaw Elias was a Tory,” Dad began. “I mean, he picked the wrong side in the Revolution.” When the Revolutionary War reached the South, Wambaw Elias, instead of joining the American rebels, went to the British commander in Charleston, Lord Cornwallis, who gave him a company of men and the rank of colonel. Wambaw Elias fought the patriots and burned their houses until such time as the British lost and his victims called for revenge. The Americans went for Wambaw Elias’s human property, dragging off some fifty slaves from Wambaw plantation, while other black workers managed to escape into the woods. Wambaw Elias knew he had no future in the United States and decided to cash in his assets. Eventually he captured the slaves who had run away, sold them, then took his family to England, where he lived for another thirty-eight years, regretting to the last that he had been forced to give up the life of a slave owner.

  In the Ball family, the tale of Wambaw Elias and his slaves passed as a children’s story.

  In my childhood, our family lived in various small towns and cities in the South. Dad was an Episcopal priest, so the houses we lived in belonged to the church, and my parents owned a single car. Throughout my spartan, God-fearing upbringing, I sensed we were different from other people. It wasn’t merely that Dad was a clergyman, though certainly that set us apart; but “our people” had once controlled a slave dynasty.

  The first piece of paper I remember my father presenting to me regarding the family was an obituary from the Charleston News and Courier. A long strip of yellowed newsprint, the clipping carried the headline “Isaac Ball, 88, Confederate, Dies.” Isaac Ball was Dad’s grandfather. He was born in 1844 on Limerick plantation, one of the many Ball tracts, and died in 1933 in Charleston. Dad used to call him Isaac the Confederate to distinguish him from the seven other Isaacs in the family tree, and because on April 28, 1862, one week after his eighteenth birthday, Isaac joined the South Carolina Militia, First Regiment, Artillery, and went to fight in the War Between the States. (The War Between the States is what the Civil War has been called in the South.) When Isaac was born, Limerick plantation, thirty miles north of Charleston, was the largest of the Ball plantations, measuring 4,564 acres. In the years before Isaac’s birth, some three hundred people lived in slavery there. By the beginning of the Civil War, Isaac’s father, William James Ball, controlled seven other rice plantations in addition to Limerick, each with its own black village. His Ball cousins and in-laws owned more land and people.

  I have several photographs of Isaac, my great-grandfather. He was tall, lean, carefully dressed, and had thin brown hair that he lost as he aged. Isaac wore a mustache and a tuft of hair below his lower lip throughout life; the hair is dark in photographs from the Civil War era and white in later pictures. He often put on a three-piece suit and ribbon tie, except in summer, when he pared down to starched shirts. Among Isaac’s pleasures was playing the violin. He was known to play Bach and, I believe, now and then a Virginia reel. In his later years, Isaac wrote poetry, though he never published it, perhaps because in his society the desire to publish was thought to be vain, a bid for attention. I have a few of his manuscripts. The verses consist of love lyrics to his wife, meditations on old things—one about a country church, another about rusting Confederate guns—and elegies about the Civil War, whose outcome caused him much sadness.

  Dad grew up in the house where Isaac spent his last twenty years, and he had strong memories of the old man. Toward the end of his life, Isaac was nearly blind from glaucoma. As Isaac was losing his sight, Dad said, he used to shave with his eyes closed, explaining that he was rehearsing the necessity of having to shave blind. In his seventies and eighties, with pinhole vision, Isaac continued to get around town and took regular walks with a cane. Near the house where he lived, at the tip of a peninsula that forms the oldest part of Charleston, there is a delicate little park called White Point Gardens. The park contains a grove of oaks with overhanging moss and an octagonal bandstand, and is framed on two sides by the waters of Charleston harbor. Isaac would feel his way to the park and find the eastern edge of the green, where a tall seawall, known as the High Battery, stands against the tides. From this place it is possible to see, far out in the water, Fort Sumter, the old defense bulwark built on an island at the entrance to the harbor. On the morning of April 12, 1861, rebel batteries around Charleston opened fire on Federal troops stationed at the fort, loudly opening the Civil War. Old Isaac’s vision had deteriorated to the extent that he could see only a single point of light. Standing on the High Battery, he would raise his cane to his shoulder like a rifle, and, aiming the stick at Fort Sumter, pretend to fire shots at the Yankees, vindicating the lost war of his youth, which robbed him of his patrimony.

  My mother, Janet Rowley, born in New Orleans, also had a plantation heritage. Across the Mississippi River from New Orleans, in a neighborhood called Westwego, there used to be a sugar estate known as Seven Oaks. The sugar fields, workers, and mansion at Seven Oaks (built in 1840) were the property of one of my mother’s maternal ancestors, Michael Zehringer. Zehringer’s grandfather had come to Louisiana in 1720 from Franconia, a section of Bavaria; later the family changed the spelling of its name to Zeringue, the better to glide through the French-speaking caste of Louisiana slave owners. A granddaughter of the master of Seven Oaks, Marie Constance Zeringue, married a man named Yves Caesar LeCorgne. Marie and Yves had a great-granddaughter, Edna LeCorgne, my mother’s mother, whom I loved.

  A yellowing photograph of the Seven Oaks mansion used to hang in the hall of our house. The picture showed a whale of a building in Greek Revival style; six two-story columns lined each of the four facades, making twenty-four Doric columns all the way around. By the time of the photograph, the plantation had long passed out of the family and stood abandoned and decrepit.

  In the family of my mother’s father, the Rowleys, were more slave owners. (Rowley is my middle name.) In 1834, Charles N. Rowley, my mother’s great-grandfather, married a Louisiana heiress, Jane Kemp Girault, who gave him control of her 2,200-acre cotton plantation, Marengo, and seventy-six slaves. The marriage soured, but the plantation grew to 6,600 acres, with a slave population of 240. Charles Rowley later went into the military, became a commissioned officer, and when the Civil War began rose to the rank of brigadier general, commanding the Sixth Brigade of the Louisiana Militia. Charles could not bear the defeat of the Confederacy; after the war, he fled the country for Brazil, leaving behind his wife and six children to join a faction of exiled Confederates. Brazil was a sugar-producing nation
where slavery would not be abolished until 1888. Maybe Charles believed that if he had remained in the United States he would have been persecuted for his role in the war, or perhaps he simply could not let go of the lifestyle of slave master. In South America, Charles Rowley evidently once again acquired human property before he died in July 1869, at age sixty-three, in the province of Rio de Janeiro.

  Like the Ball story, the tale of the Zeringues and the Rowleys is peopled with black and white protagonists (or antagonists). Although someday I may look into my Louisiana family, this story follows my father’s clan, and the lives of the thousands they enslaved.

  Although in my early childhood our family lived in other parts of the South, in summers we paid visits to South Carolina to mingle with the relatives, and when I was nine we moved to the state. Soon Dad took my brother, Theodore Jr., and me to see the old rice plantation district the Balls once knew as their neighborhood. The three of us drove north out of Charleston on a two-lane blacktop toward the upper streams of the Cooper River, where rice grew for more than two hundred years. Thick grass swallowed the edge of the asphalt, cicadas screeched in the branches, and a skyline of pine trees scored the horizon. I remember the suffocating heat that seemed to radiate up from the ground rather than descend from the sun and the air that felt like a wet cloth on the lungs. We passed unpainted cottages belonging to black families, each house with two or three rooms, a little porch, and a pitched tin roof. On one porch sat a thin old man wearing a blue workshirt.

  “There’s George,” said Dad, pulling the four-door to a stop.

  Dad directed Ted and me to stay in the car, and I seem to remember that he wore a strained look on his face as he walked across the grass, up the three wooden steps of the porch, to shake hands with frail, black George. I thought I saw him gesturing, pointing over George’s head to some mile-away place. In a moment, George and my father disappeared into the cottage. Until that day, on every occasion when my parents bumped into friends, they had introduced us, but this time Dad had left off his manners. For half an hour my brother and I sat simmering on the hot vinyl seats of the car, swatting mosquitoes. Eventually Dad emerged from the house and made his good-byes. After this encounter, the rest of the day’s tour, to an old church and down some dirt lanes, felt strange. On the way back to Charleston, Dad stayed quiet about George. We retraced our route past the empty tracts where the Balls used to rule and the flora grew in reverse, thinning and clearing. I never found out anything more about George (not even his last name), or about the mysterious thing that changed the expression on Dad’s face.

  I went to college and settled in New York, where I began writing for newspapers and magazines. Years passed, and occasionally I visited Charleston. From time to time, the “Ball book” came down off the shelf, and when it did the plantations shadowed my dreams. The Balls lived side by side with black families for six generations, but the story, as I knew it, was divided in two. On one side stood the ancestors, vivid, serene, proud; on the other their slaves, anonymous, taboo, half human. I knew a lot about the Balls, but I never knew much about the slaves, even though on the plantations black people far outnumbered white. What were their names? How did they live? Who were their loved ones? When did they leave the plantations, and where had their descendants gone? Could their families be found? But once the book went back in the bookcase, the dreams faded.

  In the mail one year came an invitation to a Ball family reunion in South Carolina. The purpose of the event, as announced by its septuagenarian organizers, was to convey the plantation story to the younger generations. Everyone, however far away, was invited home to commune with the ancestors and bathe themselves in lore. Although the Ball lands had been sold when the rice business dwindled after the end of slavery, and the fortune was long gone, documents, pictures, and above all stories remained. My memories of childhood were beginning to be released, and the invitation threw open a door in my mind. I brought out a photograph of Isaac the Confederate, Dad’s grandfather, and the faceless crowd of slaves gathered once again before my eyes.

  Despite my having left the South, the plantation past was etched in my unconscious. The prospect of the family reunion pushed me, finally, to come to terms with it. To contemplate slavery—which for most Americans is a mysterious, distant event—was a bit like doing psychoanalysis on myself. Did the plantations form part of my identity? By outward measure, no. The wealth created by the slave system was destroyed, and the latter-day Balls had no inheritance from it. Some of the family had manners, others none; some had money and status, some neither. But inwardly the plantations lived on. In childhood, I remember feeling an intangible sense of worth that might be linked to the old days. Part of the feeling came from the normal encouragements of parents who wanted their children to rise. An equal part came from an awareness that long ago our family had lived like lords, and that the world could still be divided into the pedigreed and the rootless.

  The invitation to the family reunion sat on my desk, beckoning. No one among the Balls talked about how slavery had helped us, but whether we acknowledged it or not, the powers of our ancestors were still in hand. Although our social franchise had shrunk, it had nevertheless survived. If we did not inherit money, or land, we received a great fund of cultural capital, including prestige, a chance at education, self-esteem, a sense of place, mobility, even (in some cases) a flair for giving orders. And it was not only “us,” the families of former slave owners, who carried the baggage of the plantations. By skewing things so violently in the past, we had made sure that our cultural riches would benefit all white Americans.

  The subject of the plantations stirred conflicting emotions. I felt proud (how rare the stories!) and sentimental (how touching the cast of family characters!). At the same time, the slave business was a crime that had not fully been acknowledged. It would be a mistake to say that I felt guilt for the past. A person cannot be culpable for the acts of others, long dead, that he or she could not have influenced. Rather than responsible, I felt accountable for what had happened, called on to try to explain it. I also felt shame about the broken society that had washed up when the tide of slavery receded.

  I decided I would make an effort, however inadequate and personal, to face the plantations, to reckon with them rather than ignore their realities or make excuses for them. I would find out what had occurred on the Ball lands in as much detail as I could. America was beginning to work through the tragic parts of its history, and the Ball name, once admired in a small corner of the country, seemed likely to become a casualty of that process, painted with infamy. Paradoxically, by describing as honestly as possible what the family had done, I might clear some space around our name, and around us.

  When finally I chose to look into the slave past, I felt a remarkable calm, and the rest of the path seemed clear. To complete the legacy, I would try to find descendants of the slaves. The plantation heritage was not “ours,” like a piece of family property, and not “theirs,” belonging to black families, but a shared history. The progeny of slaves and the progeny of slave owners are forever linked. We have been in each other’s lives. We have been in each other’s dreams. We have been in each other’s beds. As I prepared to go back to South Carolina, I thought we should meet, share our recollections, feelings, and dreams, and make the story whole.

  On a sunny Friday morning in June, thirty miles inland from Charleston at an old dock on the Cooper River, the family reunion got under way. Some 150 relatives from around the state and scattered parts of the country climbed aboard a chartered boat for a tour of the river where the Ball plantations once stood and where their remnants still moldered on the banks. Many wore shorts and T-shirts in the Carolina heat, though a few women turned out in heels and jewelry. The youngest was less than a year, the eldest eighty-eight. Reeds brushed against the hull of the double-deck boat and greetings and laughter echoed over the water as each Ball walked the gangplank and made a little jump onto the deck. My brother, Ted, and his wife, Pam Taylor,
came from Louisiana.

  The engine turned over and we took our seats. In a moment the skipper rounded a bend in the river and we came in sight of the oldest Ball place, Comingtee plantation. It was here that the founder of the family, the first Elias Ball, came from England to live in the year 1698. (The name Elias Ball would be used for many future sons, including Wambaw Elias, the Tory who moved back to England two generations later.) The ruined brick hulk of the rice mill at Comingtee stood within plain sight. Sticking out of the mud near the riverbank were pieces of rotting wood shaped like the headboards of beds—the sluices that once controlled the flow of water onto the rice fields.

  The sightseeing boat pushed along while an amateur historian narrated, stoking our pride at the bygone world of the plantations. The size of the Ball lands once ranged from small three-hundred-acre tracts, worked by thirty or forty field hands, to giant estates of three or four thousand acres, with hundreds of slaves. Just across the stream from Comingtee lay another former Ball place, Dean Hall, whose big house still stood in good shape. Some of the South’s plantations were now used as second homes by wealthy families, but many had been swept away. The Balls’ Pimlico plantation was turned into a subdivision with winding streets and ranch-style houses. A few miles up the river lay two old Ball tracts, Kensington and Hyde Park, whose buildings were mostly gone. Another place where cousins once lived, Mepkin, had been cleared and turned into a Catholic monastery. A forestry company had bought much of the family land and planted pine trees. We peered out over the water and murmured at our broken heritage.

  In the slave days the rice fields lay at the edge of the river. Each plot was surrounded by a “rice bank,” an earthen levee several feet high and perhaps two hundred yards long on four sides of the field. The remnants of rice banks, eroded by the current, lay submerged in the water, presenting obstacles for boats, and a hull with a draft of more than a couple of feet could easily run aground. As the double-decker headed for another bend, it lurched and teetered on one of the rice banks. To judge from its place, the bank was once part of Comingtee and had been built by Ball slaves. The pilot tried to dislodge his vessel, stuck in the muck where the slaves had worked, but the helpless engine made hoarse guttural sounds. Half an hour passed, then an hour.