Types of Primary Sources
Database APIs & CONTENTdm
DO NOT SCRAPE LIBRARY DATABASES
Why? You're stealing protected information, and that violation of Terms of Service can get access turned off for the entire campus.
But, there are options!
Always ask your library liaison if you don't know how to access the underlying data in a library database -- we can help you do it legally! Depending on the database vendor, we may be able to get hard drives of all the underlying data, or we can help to set up API access for you.
- Research Consultations with Librarians by SubjectClick on a librarian's name to find contact and schedule information.
- About CONTENTdm APIsInstructions on how to use the CONTENTdm API.
- Carleton CONTENTdm Digital CollectionsThere are many digital collections hosted in CONTENTdm, including Course Catalogs, Northfield historical material, and other digital objects used in classes.
TV News
- Internet Archive: TV News archiveSearches news captions for most major networks going back to 2009.
- GDELT Television APIUses the Internet Archive TV Archive data. Supported by Google Jigsaw.
- News Data Service PowerSearch This link opens in a new windowSee the Research/IT Desk - Ask a Librarian to log you in. News Data Services provides access to broadcast news transcripts from local and national tv and radio stations from 2002 to today. Streaming video content is available for 60 days from the time of broadcast and can be clipped and saved for later analysis.
Twitter access
- Twitter API TutorialNOTE: You are not able to retrieve tweets older than 7 days.
- DocNow Catalog of Twitter datasetsThe DocNow Catalog is a collectively curated listing of Twitter datasets. Public datasets are shared as Tweet IDs, which can be hydrated back into full datasets using our Hydrator desktop application.
- Internet Archive: Global Events ArchiveThese collections of global events have been created by the Archive-It team in conjunction with curators and subject matter experts from institutions around the world.
Using git
- GitHub
This link opens in a new window
- GitHub DesktopIf you are not going to be using the command line (on any operating system), this is the recommended user interface.
- Pro Git: Everything You Need to Know About Git (2022)
An open access textbook with a deep dive into how to get started with git on the command line.
