History of Integrated Data Systems (IDS)

The Book of Life

Shortly after the end of World War II, Halbert L. Dunn, recognized by many as a key founder of the U.S. vital statistics system, published an article on linking an individual’s administrative records across the course of his life[1]. Dunn referred to this, rather Biblically, as the “Book of Life.” In the article, he emphasizes its importance to the individual, noting that the primary benefit to government agencies lies in better understanding the individuals they serve, which leads to better programs and outcomes for U.S. citizens. Dunn even goes into ways to link (or “bind,” as he refers to it) each citizen’s “Book of Life” into one volume, and provides an example of how Canada, upon legislative order, had already designed and carried-out a method linking all vital records for each Canadian citizen under 20 years of age.

[1]   Special thanks goes to Emily Putnam-Hornstein and her colleagues at the Children’s Data Network for unearthing this important piece of history.
Dunn, H. L. (1946). Record linkage*. American Journal of Public Health and the Nations Health36 (12), 1412-1416.

Pete Bailey & the South Carolina Integrated Data System (IDS)

While Dunn did a great deal to advance the use of data to inform program and policy analysis—The National Association for Public Health Statistics and Information Systems even named its most prestigious award in his honor—it was Pete Bailey, a statistician based in South Carolina, that really brought Dunn’s concepts to fruition. Dunn was influential in the creation of administrative reporting systems, and Bailey found ways to link the administrative data from such systems across the life course—effectively, creating the bound “Book of Life” Dunn proposed in his 1946 article.

Bailey, one of 14 children, was born in rural Alabama. He grew up in poverty, and was determined to build a better life not only for his family, but also for the countless other families that were born into similar situations. He knew that, ultimately, government agencies needed to determine a more effective way to express both concern and care for their citizens, and decided that the first necessary step was getting these agencies to communicate regularly with each other.

Bailey was given the opportunity to realize this dream in the early 1970s when Governor John West of South Carolina hired him, a young but already accomplished statistician, to develop a health statistics section within the governor’s office to inform policy across the state. Bailey worked tirelessly to form collegial relationships with staff members in all state agencies. This allowed him to build the trust necessary to integrate data across systems in order to better understand how policies and programs instituted by one agency affected citizens across the board.

Bailey’s organization was eventually moved out of the governor’s office to the South Carolina Budget and Control Board, and, until the 1990s, was used to integrate administrative data from multiple state agencies in order to better inform state-level policy decisions. In the 1990s Bailey’s organization experienced rapid growth due to two events: (1) a grant from the Robert Wood Johnson Foundation and (2) the passage of a state statute requiring administrative data to be integrated. Currently, South Carolina’s IDS integrates administrative data across more than 20 agencies, and is widely considered to be the leading expert on IDS.

[2] Pete Bailey authored an invaluable report on developing an IDS for AHRQ.

AISP – Pete Bailey discusses South Carolina’s Integrated Data Systems from AISP_UPENN.

The Early to Mid-90s

Other states (as well as counties and cities) eventually followed in Bailey’s footsteps, and often relied on his expertise in building and using IDS to better serve their jurisdiction’s citizens. In Chicago, Chapin Hall established an IDS to continue to build upon its long-standing mission to enhance child well-being. Meanwhile, in nearby Cuyahoga County, OH, Claudia Coulton and colleagues developed CHILD, an IDS focused on improving child health and well-being. Washington State’s IDS grew out of a mid-1990s needs-assessment. Michigan’s system, which originated around the same time as Washington State’s, was developed to cultivate cross-agency government collaborations. Smaller efforts also began to emerge across the United States during this time. For example, in the mid-1990s, Dennis Culhane began working with administrative data in Philadelphia and New York to better understand the patterns of homelessness.

The Turn of the 21st Century

Allegheny County PA’s IDS officially launched in 1999, and was primarily focused on using integrated administrative data for case management purposes. In order to create this IDS, Marc Cherna, Department of Human Services’ Director, first consolidated several social service departments under one umbrella to create a more comprehensive human services system in Allegheny County. This action was taken in response to a series of very public scandals involving Allegheny County’s child welfare system. After this process was complete, he developed an IDS to link administrative data across the different DHS agencies. This quickly proved to be essential to increasing positive outcomes for citizens receiving services from Allegheny County DHS, and soon grew to also integrate administrative data for policy analysis purposes.

In 2000 Culhane joined forces with two other researchers at the University of Pennsylvania (Penn), John Fantuzzo and Trevor Hadley, to develop the Kids Integrated Data System (KIDS) in order to provide decision-makers in Philadelphia with timely, cost-effective information. The mission of KIDS was to address linking administrative data through a systematic (vs. ad hoc) approach in order to provide a comprehensive view of how policies and programs affect the individuals they are intended to serve in a time- and cost-effective manner.

Paul Stiles, after completing a post doc at Penn, collaborated with John Petrila to build the Policy and Research Services Data System (PSRDC) system at the University of South Florida in order to enhance health and human services’ programs and policies. Meanwhile, in Los Angeles County, Manuel Moreno and Halil Toros were building the Enterprise Linkages Project (ELP)  in order to address the county service use of individuals experiencing chronic homelessness. New York City followed shortly thereafter, developing an IDS in order to provide policy-makers with more accurate information on interagency policy areas.


The John D. and Catherine T. MacArthur Foundation, largely due to their longstanding commitment to housing, became interested in Culhane and Fantuzzo’s work with KIDS—especially with regards to its replicability outside of Philadelphia. This interest eventually led to the formation of the Actionable Intelligence for Social Policy (AISP) Network.

Recent Sites

Recent sites that have developed mature IDS and joined the AISP Network: