Philadelphia Social History Project

Essay

The Philadelphia Social History Project (PSHP) was a large and ambitious interdisciplinary project that played a central role in transforming the study of urban and social history. From its inception in 1969 until the project’s closure in 1985, the PSHP employed dozens of research associates and computer programmers, as well as hundreds of undergraduate students, to create a computerized database to study the effect of industrialization and urbanization on Philadelphia’s minority populations between 1838 and 1880.

Under the direction of Theodore Hershberg (b. 1941) at the University of Pennsylvania, the PSHP received $78,000 from the Center for Studies of Metropolitan Problems of the National Institute of Mental Health in 1969 to create machine-readable data and develop automated processes to study rates of social mobility of the African American, Irish, and German populations in Philadelphia. Building on research models developed by scholars such as Stephan Thernstrom, and responding in part to increased public awareness of civil rights issues in the United States, Hershberg and PSHP researchers sought to determine, among other topics, whether the African American socioeconomic experience was unique or whether it shared characteristics with Philadelphia’s other large minority ethnicities.

The Moore School building at the University of Pennsylvania.
The Moore School building at the University of Pennsylvania, where many of the software engineers who worked on the PSHP were based.

Beginning with data from the Pennsylvania Abolition Society and the Society of Friends manuscript censuses from 1838, 1847, and 1856, and supplementing this with data from the U.S. population censuses of 1850, 1860, 1870, and 1880, Hershberg and his associates developed ways to convert tens of thousands of records into machine-readable form and input these into a large database. Computer programmers, many from the Graduate Group in Computer and Information Science at the Moore School at the University of Pennsylvania, wrote sophisticated programs so that the data could be manipulated and analyzed by PSHP research associates. With additional funding eventually totaling more than $3 million from NIMH and other agencies, the PSHP database was expanded to include census information about native-born white Americans, as well as data from U.S. Census manufacturing schedules, mortality registers, city business directories, and lists of institutions and voluntary associations.

New Strides in Linking Data

Beyond the creation of a large, comprehensive database, PSHP’s greatest accomplishments may have been the processes its researchers and programmers developed to assign geographic coordinates to data and to link disparate records within the database. By placing every person and institution within a specific map “grid square” (a fixed area, usually 1 x 1¼ city blocks), Hershberg and his associates could study the city and its population in ways, and at scales, that previously had been impracticable. For instance, by associating data about transportation networks with grid coordinates, researchers could compare “journeys-to-work” of different communities and better understand how transportation affected the social and spatial organization of the city.   Sophisticated computer programs added breadth and depth to PSHP studies because linkages could be made to match individuals, industries, or associations from multiple data sources. PSHP researchers could, with reasonable certainty, match individuals and businesses, even those whose names or addresses had changed, across data sets from different years.

An 1859 photograph showing row homes on Chestnut Street near Fifth.
The Philadelphia Social History Project used census records and other data to create a database of the population of nineteenth-century neighborhoods such as this one, on Chestnut near Fifth Street, in 1859. (Library Company of Philadelphia)

PSHP was one of the earliest, and largest, projects in the humanities to employ a computerized database to conduct its research. It drew researchers from multiple disciplines including history, demography, ethnic studies, urban history, economics, and sociology and operated on an unprecedented scale of staffing, funding, and ambition. It produced massive amounts of statistical information and developed innovative and sophisticated programs to manipulate and analyze data. But while the project was frequently hailed as transformative because of its interdisciplinary focus and because of the multiple uses to which its database could be put, it also was criticized for overemphasizing quantitative, rather than qualitative, data analysis. Critics also called attention to the project’s comparatively small output of published scholarship (relative to its considerable funding) and inaccuracies in the database (in particular, categorization of occupational and industrial data from the 1900 and 1930 censuses).

Hershberg left the Philadelphia Social History Project in 1981 and later founded the Center for Greater Philadelphia at the University of Pennsylvania. The PSHP continued to operate until its funding ended in 1985. Many of the electronic data sets were retained by PSHP research associates. The paper and microfilm archive, containing published and unpublished reports, system manuals, program and tape documentation, administrative records, maps, printouts of tabulated information from PSHP data sets, and photocopies of census schedules, was transferred to the University of Pennsylvania Libraries.

Regan Kladstrup is the Assistant Director of the Special Collections Processing Center at the Kislak Center for Special Collections, Rare Books and Manuscripts, University of Pennsylvania Libraries.

Copyright 2015, Rutgers University

Gallery

The Moore School at the University of Pennsylvania

PhillyHistory.org

The Moore School building at the University of Pennsylvania, in West Philadelphia, held the Graduate Group in Computer and Information Science during the years that the Philadelphia Social History Project was active. Many of the computer scientists who worked on the project originated at this school. The programs they developed for use on the project allowed for unique links to be formed between data sets, enabling individuals to be traced across the data for multiple years, resulting in deeper analysis of the data.

Fifth and Chestnut Streets, 1859

Library Company of Philadelphia

This 1857 photograph shows the neighborhood around the corner of Chestnut and Fifth Streets, which contained both residences and businesses. Neighborhoods such as this were included in the Philadelphia Social History Project, in which Theodore Hershberg and his associates used data from the Pennsylvania Abolition Society, Society of Friends and U.S. Census, among other sources, to generate a database reflecting the population of neighborhoods. Though it was focused on ethnic minorities and specifically the African American community, the PSHP was eventually expanded to include information about native-born whites after receiving grants from the National Institute of Mental Health and other organizations.

Related Topics

Themes

Time Periods

Locations

Essays

Related Reading

Conk, Margo. “Industrial Philadelphia,” review of Philadelphia: Work, Space, Family, and Group Experience in the Nineteenth Century, Essays Toward an Interdisciplinary History of the City, ed. by Theodore Hershberg, Pennsylvania Magazine of History and Biography, July 1, 1982 (vol. 106, no. 3): 423-31.

Hershberg, Theodore. “The Philadelphia Social History Project: A Methodological History.” Ph.D. diss., Stanford University, 1973.

Hershberg, Theodore, ed. Philadelphia: Work, Space, Family, and Group Experience in the Nineteenth Century, Essays Toward an Interdisciplinary History of the City. Oxford: Oxford University Press, 1981.

Hershberg, Theodore. “Interdisciplinary Research at the Philadelphia Social History Project: Analytic Goals, Data and Data Manipulation Strategies for the Study of the Nineteenth-Century Industrial City.” In Clubb, Jerome M. and Erwin K. Scheuch, eds., Historical Social Research: the Use of Historical and Process-Produced Data. Stuttgart: Klett-Cotta, 1980. (Historisch-Sozialwissenschaftliche Forschungen: quantitative sozialwissenschaftliche Analysen von historischen und prozess-produzierten Daten, Bd. 6).

Hershberg, Theodore, et al. “The Philadelphia Social History Project,” Historical Methods Newsletter special issue, v.9, no.2-3 (March-June 1976).

Lardner, James. “History by Numbers: Defending Computers as Contemporary Tool.” The Washington Post, March 9, 1982.

Palladinetti, Stephen J. “The Evolution of the Source Conversion Process at the Philadelphia Social History Project.” M.S. thesis, University of Pennsylvania, 1974.

Selected PSHP Publications

Furstenberg, Frank Jr., Theodore Hershberg, and John Modell. “The Origins of the Female-Headed Black Family: The Impact of the Urban Environment,” Journal of Interdisciplinary History, vol. 6, no. 2 (September 1975), 211-33.

Glassberg, Eudice. “Work, Wages and the Cost of Living: Ethnic Differences and the Poverty Line, Philadelphia, 1880.” Pennsylvania History, vol. 46, no. 1 (January 1979), 17-58.

Haines, Michael. “Fertility and Marriage in a Nineteenth-Century Industrial City: Philadelphia, 1850-1880,” Journal of Economic History, vol. 40, no. 1 (March 1980), 151-158.

Hershberg, Theodore. “Free Blacks in Antebellum Philadelphia: A Study of Ex-Slaves, Free-Born, and Socioeconomic Decline,” Journal of Social History (December 1971), 183-209.

Hershberg, Theodore. “A Method for the Computerized Study of Family and Household Structure Using the Manuscript Schedules of the U.S. Census of Population, 1850-1880,” The Family in Historical Perspective, no. 3 (Spring 1973), 6-20.

Laurie, Bruce, Theodore Hershberg, and George Alter. “Immigrants and Industry: The Philadelphia Experience, 1850-1880,” Journal of Social History, vol. 9, no. 2 (Winter 1975), 219-248.

Laurie, Bruce. Working People of Philadelphia, 1800-1850. Philadelphia: Temple University Press, 1980.

Modell, John, Frank F. Furstenberg Jr., and Theodore Hershberg. “Social Change and Transitions to Adulthood in Historical Perspective,” Journal of Family History, vol. 1, no. 1 (September 1976), 7-32.

Seaman, Jeff, and Gretchen Condran. “Nominal Record Linkage by Machine and Hand: An Investigation of Linkage Techniques Using the Manuscript Census and the Death Register, Philadelphia, 1880,” 1979 Proceedings of the Social Statistics Section of the American Statistical Association, 678-683.

Related Collections

Philadelphia Social History Project Records, University of Pennsylvania Libraries, Van Pelt-Dietrich Library Center, 3420 Walnut Street, Philadelphia.

Philadelphia Social History Project Series, Interuniversity Consortium for Political and Social Research (ICPSR), 330 Packard Street, Ann Arbor, Mich.

Backgrounders

Links

Connecting the Past with the Present, Building Community, Creating a Legacy