Syllabus Resources

 

Journalism 772 and 472: Computer-Assisted Reporting

Ira Chinoy / Philip Merrill College of Journalism, University of Maryland

 

DATA ON THE WEB

A sample of sources

 

This site has examples of data that can be downloaded from the Web or searched on the Web

 

Lists of links to data sources on the web:

  Downloadable Data on the Net (NICAR Net Tour): http://www.ire.org/training/nettour/papertrails.html

  A Journalists Database of Databases (Drew Sullivan): http://www.drewsullivan.com/database.html

  Finding Data on the Internet (RobertNiles.com): http://nilesonline.com/data/

  FedStats Government site with links to over 100 federal agencies with data on the web: http://www.fedstats.gov/

  GPO Access (Government Printing Office): Search multiple federal databases from one page: http://www.gpoaccess.gov/multidb.html

 

Governments with sites for downloadable data:

Federal government: Data.gov: http://www.data.gov/

District of Columbia: Data Catalog: http://data.dc.gov/

 

Governments with sites for summary data:

Maryland: StateStat: http://www.gov.state.md.us/statestat/

Baltimore: CitiStat: http://www.baltimorecity.gov/Government/AgenciesDepartments/CitiStat.aspx

 

 

Business

  Maryland Department of Assessment and Taxation:

o   List of searchable databases: http://www.dat.state.md.us/

o   Business Data Search: http://sdatcert3.resiusa.org/ucc-charter/

o   Real Property (real estate) Data Search: http://sdatcert3.resiusa.org/rp_rewrite/

 

Culture and entertainment

  Cultural Policy & The Arts National Data Archive: http://www.cpanda.org/

  UNESCO Institute for Statistics: http://www.uis.unesco.org/ev.php?URL_ID=5275&URL_DO=DO_TOPIC&URL_SECTION=201

  Federal Communications Commission Registered cable communities: http://www.fcc.gov/mb/engineering/liststate.html

 

Demographics and social issues, current and historical:

  Census:

o   2000 Census Data for Maryland: http://www.census.gov/census2000/states/md.html

o   Historical statistics from census data: http://www.census.gov/statab/www/minihs.html

  Kids Count Annie E. Casey Foundation compilation of data on the well-being of children: http://www.aecf.org/kidscount/

o   Overview: http://www.aecf.org/kidscount/databook/

o   Downloadable raw data: http://www.aecf.org/cgi-bin/kc.cgi?action=newdata

o   Kids Count Census data online: http://www.aecf.org/cgi-bin/aeccensus.cgi?action=dataresults&area=24S

  The Association of Public Data Users: http://www.apdu.org/

  Inter-university Consortium for Political and Social Research: http://www.icpsr.umich.edu/access/index.html

  The Urban Institute:

o   State Database: http://www.urban.org/Content/Research/NewFederalism/Data/StateDatabase/StateDatabase.htm

o   National Survey of Americas Families: http://www.urban.org/Content/Research/NewFederalism/NSAF/Overview/NSAFOverview.htm

  National Center for Children in Poverty: http://nccp.org/wizard/wizard.cgi

 

Economy, business, financial institutions, nonprofits, workplace:

  Internal Revenue Service Tax Stats Nonprofits, by state: http://www.irs.gov/taxstats/charitablestats/article/0,,id=97186,00.html

  U.S. Bureau of Labor Statistics: http://stats.bls.gov/data/home.htm

o   National Compensation Survey Tables: http://www.bls.gov/ncs/home.htm#tables

o   Worker Fatalities Data: http://www.bls.gov/iif/oshcfoi1.htm

  U.S. Census Bureau E-Commerce Statistics: http://www.census.gov/eos/www/ebusiness614.htm

  National Credit Union Administration call reports: http://www.ncua.gov/data/FOIA/foia.html

  Federal Deposit Insurance Corporation - custom data downloads: http://www2.fdic.gov/sdi/main.asp

  U.S. Occupational Safety and Health Administration:

o   Searchable database of workplace injuries: http://www.osha.gov/pls/imis/establishment.html

  Maryland Department of Assessments and Taxation   

o   Searchable databases: http://sdatcert3.resiusa.org/ucc-charter/CharterSearch_f.asp

Education:

  College Results Online data on colleges and universities compiled by The Education Trust: http://www.collegeresults.org/

  Maryland School Performance Report Card: http://www.msp.msde.state.md.us/rawdata/index.asp

  Education Week student achievement: http://www.edweek.org/sreports/qc02/reports/achieve-t1.htm

  National Center for Education Statistics

o   Common Core of Data Education Data Files: http://nces.ed.gov/ccd/ccddata.asp

o   Demographics of school districts: http://nces.ed.gov/surveys/sdds/downloadmain.asp

o   Digest of Education Statistics, 2002: http://nces.ed.gov/programs/digest/d02/list_tables.asp

  U.S. Department of Education, Federal Student Aid school default rates: http://www.ed.gov/offices/OSFAP/defaultmanagement/cdr.html

  UNESCO Institute for Statistics: http://www.uis.unesco.org/ev.php?URL_ID=5275&URL_DO=DO_TOPIC&URL_SECTION=201

 

Environment, weather, natural disasters and other emergencies:

  Historical Severe Weather Data NOAA: http://www.spc.noaa.gov/archive/

  Right-to-Know Network Environmental Databases: http://www.rtk.net/rtkdata.html

  Toxicology Data Network TOXNET: http://toxnet.nlm.nih.gov/

  U.S. Environmental Protection Agency:

o   EPA Data: http://www.epa.gov/epahome/Data.html

o   EPA Envirofacts: http://www.epa.gov/enviro/

o   EPA Drinking water data: http://www.epa.gov/safewater/data/getdata.html

o   Engangered Species Protection Program Databases: http://www.epa.gov/espp/database.htm

  National Response Center (spills and other environmental discharges):

o   Home page: http://www.nrc.uscg.mil/nrchp.html

o   Downloadable data (in Excel): http://www.nrc.uscg.mil/download.html

o   Searchable data: http://www.nrc.uscg.mil/foia.html (slow)

 

Government spending:

  Office of Management and Budget: searchable databases of contracts and grants -- http://www.fedspending.org/

  Census: Federal, State, and Local Government Data: http://www.census.gov/govs/www/

  Consolidated Federal Funds Report: http://www.census.gov/govs/www/cffr.html

  Federal Assistance Award Data System: http://www.census.gov/govs/www/faads.html

  D.C. Office of Contracting and Procurement (OCP) searchable database: http://app.ocp.dc.gov/RUI/information/awards/all_awards.asp?radio=3

 

Health:

  American Public Human Services Association Federal Data Index: http://www.aphsa.org/links/federaldata.asp

  Kaiser Family Foundation / State Health Facts Online: http://www.statehealthfacts.kff.org/cgi-bin/healthfacts.cgi?action=rawdata

  Excluded Individual and Entities, Office of the Inspector General, U.S. Dept. of Health and Human Services:

o   http://oig.hhs.gov/fraud/exclusions/listofexcluded.html

  Medicare databases: http://www.medicare.gov/Download/DownloadDB.asp [Note: site was inaccessible 5-23-04]

o   comparing nursing homes (including inspections): http://www.medicare.gov/NHCompare/Static/Related/DownloadDB.asp?dest=NAV|Home|Resources|DownloadDatabase#TabTop

        Notes about the data: http://www.medicare.gov/NHCompare/Static/Related/DataCollection.asp?dest=NAV|Home|DataDetails|DataCollection#TabTop

o   home health care: http://www.medicare.gov/hHCompare/Static/Related/DownloadDB.asp?dest=NAV|Home|Resources|DownloadDatabase#TabTop

o   dialysis facilities: http://www.medicare.gov/Dialysis/Home.asp

  Centers for Medicare and Medicaid Services: http://www.cms.hhs.gov/researchers/

  U.S. Health Resources and Services Administration: Geospatial Data Warehouse: http://datawarehouse.hrsa.gov/default.htm

  Manufacturer and User Facility Device Experience Database - (MAUDE): http://www.fda.gov/cdrh/maude.html

  Centers for Disease Control and Prevention:

o   CDC Data and Statistics: http://www.cdc.gov/node.do/id/0900f3ec8000ec28

o   National Center for Health Statistics: http://www.cdc.gov/nchs/

  U.S. National Library of Medicine links to data sets and data sources: http://www.nlm.nih.gov/nichsr/hsrsites.html

  National Practitioner Databank: http://www.npdb-hipdb.com and http://www.npdb-hipdb.com/publicdata.html

  US Dept. of Agriculture Food and Nutrition Service: http://www.fns.usda.gov/pd/

o   Food Stamp Program: http://www.fns.usda.gov/pd/fspmain.htm

o   Child Nutrition: http://www.fns.usda.gov/pd/cnpmain.htm

o   WIC (Women, Infants and Children): http://www.fns.usda.gov/pd/wichome.htm

o   Food Distribution Program: http://www.fns.usda.gov/pd/fdpmain.htm

 

Housing:

  U.S. Department of Housing and Urban Development:

o   Data sets: http://www.huduser.org/datasets/pdrdatas.html

o   Subsidized housing: http://www.huduser.org/datasets/assthsg/statedata98/index.html

o   The Office of Inspector Generals Exclusion Program: http://oig.hhs.gov/fraud/exclusions.html

  Maryland Real Estate Values:

o   Searchable database: http://sdatcert3.resiusa.org/rp_rewrite/

 

International issues:

  World Trade Organization: http://www.wto.org/english/res_e/statis_e/statis_e.htm

  United Nations Statistics Division Social Indicators: http://unstats.un.org/unsd/demographic/social/default.htm

  U.S. Census International Programs Center: http://www.census.gov/ipc/www/

  International Trade Administration (US Dept. of Commerce): http://www.ita.doc.gov/td/industry/otea/

  Organisation for Economic Co-operation and Development:

o   Statistics and databases: http://www.oecd.org/statsportal/0,2639,en_2825_293564_1_1_1_1_1,00.html

  Government of India a sampling of sites with data that can be downloaded:

o   Planning Commission: http://planningcommission.nic.in/data/dataf.htm

o   Ministry of Statistics and Programme Implementation:

        Home page: http://mospi.nic.in/

        Economic Census, 1998: http://mospi.nic.in/mospi_ec.htm

o   Election Commission of India: http://www.eci.gov.in/ElectionResults/ElectionResults_fs.htm

(includes links to data in zipped Access files)

 

Law enforcement, courts and crime:

  U.S. Department of Justice Bureau of Justice Statistics:

o   www.ojp.usdoj.gov/bjs/

o   http://www.ojp.usdoj.gov/bjs/dtd.htm

o   http://www.ojp.usdoj.gov/bjs/dtdata.htm#index

  Federal Bureau of Investigation Uniform Crime Reports: http://www.fbi.gov/ucr/ucr.htm

  National Archive of Criminal Justice Data: www.icpsr.umich.edu/NACJD/

  Office of Postsecondary Education Campus Security Statistics: http://ope.ed.gov/security/

  Maryland Judiciary Case Search [search engine]: http://casesearch.courts.state.md.us/inquiry/inquiry-index.jsp

 

Military

  Windfalls of War The Center for Public Integritys report on military contractors: http://www.publicintegrity.org/wow/

  Department of Veterans Affairs http://www.va.gov/vetdata/index.htm

  Federation of American Scientists Arms Sales Monitoring Project: http://www.fas.org/asmp/profiles/index.html

o   Sample data set from queries:

        U.S. military aid appropriations for all countries for 2003 -- http://www.fas.org/asmp/profiles/aid_db.php?regionin=%&ctryin=%&fy1in=2003&fy2in=2003&appin=1

        U.S. military aid deliveries to all countries in the Middle East and South Asia for 1990-2001: http://www.fas.org/asmp/profiles/aid_db.php?regionin=nesa&ctryin=%&fy1in=1990&fy2in=2001&appin=0

 

Politics:

  Federal Election Commission

o   Downloadable databases: http://www.fec.gov/finance/disclosure/ftp_download.shtml

        Detailed files: http://www.fec.gov/finance/disclosure/ftpdet.shtml#a2005_2006

        Summary files: http://www.fec.gov/finance/disclosure/ftpsum.shtml

        Tutorial on importing FEC data into Microsoft Access: http://www.fec.gov/finance/disclosure/working_with_data_files.pdf

o   Searchable databases:

        Individual contributors: http://www.fec.gov/finance/disclosure/norindsea.shtml

  Campaign Finance Information Center (c/o IRE) data sources for state races: http://www.campaignfinance.org/linksstate.html

  The Buying of the President 2004 The Center for Public Integritys report on candidate finances: http://www.bop2004.org/bop2004/dw.aspx

  Follow the Money The Institute on Money in State Politics: http://www.followthemoney.org/database/power_search.phtml

  The Center for Responsive Politics: www.opensecrets.org

 

Real estate:

  Maryland Department of Assessment and Taxation -- Real Property (real estate) Data Search: http://sdatcert3.resiusa.org/rp_rewrite/

 

Science and Technology:

  National Science Foundation Division of Science Resources Statistics: http://www.nsf.gov/sbe/srs/stats.htm

  UNESCO Institute for Statistics: http://www.uis.unesco.org/ev.php?URL_ID=5275&URL_DO=DO_TOPIC&URL_SECTION=201

 

Sports and recreation

  Hunting accidents Maryland Natural Resources Police: http://www.dnr.state.md.us/nrp/huntingaccidents.html

  Sports salaries USA Today database:

o   Baseball: http://asp.usatoday.com/sports/baseball/salaries/default.aspx

o   Basketball: http://asp.usatoday.com/sports/basketball/nba/salaries/default.aspx

o   Football: http://asp.usatoday.com/sports/football/nfl/salaries/default.aspx?Loc=Vanity

o   Hockey: http://asp.usatoday.com/sports/hockey/nhl/salaries/default.aspx

 

Transportation:

  National Highway Traffic Safety Administration:

o   Fatality Analysis Reporting System (FARS): http://www-fars.nhtsa.dot.gov

o   Office of Defects Investigations: http://www-odi.nhtsa.dot.gov/cars/problems/defect/defectsearch.cfm

  Office of Hazardous Materials Safety, U.S. Dept. of Transportation: http://hazmat.dot.gov/pubs/inc/hmisframe.htm

  Federal Rail Administration Office of Safety Analysis: http://safetydata.fra.dot.gov/officeofsafety/Downloads/Default.asp

  Bureau of Transportation Statistics National Transportation Atlas database: http://www.bts.gov/publications/national_transportation_atlas_database/2006/

  Intermodal Transportation Database: www.transtats.bts.gov

  National Transportation Safety Board - Aviation Accident Database: http://www.ntsb.gov/NTSB/query.asp (MS Access data)

  Boating accidents Maryland Natural Resources Police: http://www.dnr.state.md.us/nrp/boatingaccidents.html

  Federal Transit Administration National Transit Database: http://www.ntdprogram.com/NTD/ntdhome.nsf/?Open

  National Aviation Safety Data Analysis Center: http://www.nasdac.faa.gov/

 

 

Mechanics of data importing

 

Help with data importing and data formats:

  Tip sheet on data formats (Jeff South, VCU): Link: (use class username and password)

  Tip sheets on importing text files containing data, such as files ending in *.txt, *.csv, *.dat, *.tab (you can also use the Help system in Excel and Access, and both programs have data import wizards; the tip sheets below explain how to get to the wizards and give you anoverview):

o   Importing into Excel (for record sets no larger than 65,536 rows, including field names and totals)

        Overview of import wizard: http://www.edmond.k12.ok.us/online_tutorials/tutorials/excel_2002_a/excel_2002_advanced_manual_educational_webp4.htm

        Importing delimited files: http://biology.wsc.ma.edu/biology/experiments/exceldownload.html

o   Importing into Access:

        Overview of importing delimited data: http://www.billianshealthdata.com/Dispstpg.htm?CD=2&ID=51

  SAS System Viewer free application for viewing and printing SAS files: http://www.sas.com/apps/demosdownloads/setupcat.jsp;jsessionid=967763786440F8E958BC0E4F884805EE.tomcat4?cat=SAS+System+Viewer

  Converting from PDF to TEXT files: Sometimes data appears in PDF files and cannot be directly imported into database or spreadsheet programs as they are. However, depending on the way the PDF file was created, it may be possible to convert its contents into a text file that can then be imported into Excel or Access. You can read more about this here in a tip sheet from the National Institute for Computer-Assisted Reporting : http://www.ire.org/training/nettour/pdf/PDFTOTEXT.pdf