Hey! Links to the TV Search API provided by BBC Snippets
You can register for you own Mashery API key to access the service like this, or use the demo one for News Hack “ca7tqq7nqbk4c2wu3uewse7a”:
More details below
Search for any word spoken in any BBC TV show …
The Snippets search API provides a simple way for developers to query programme information and subtitles for every BBC TV show broadcast since June 28th 2007.
The API has been built on top of the subtitle files provided by BBC Redux . Because of the way Redux captures the Freeview subtitle metadata it’s possible to extract them using OCR software. Once extracted they can be treated as verbatim transcripts (see below for more details), indexed to make them queryable and mapped to Redux and PIPs programme metadata. This means we’re now able to offer an API that allows complex queries spanning both systems.
Quick Start
A simple search query
http://bbc.api.mashery.com/snippets/search?keywords=hello&api_key=aaabbbccc123123123
{ "count":10840, "results":[ { "channel_name":"bbcnews24", "date_facet":{ "day":"21", "hour":"19", "min":"00", "month":"02", "week":"2011-08", "year":"2011", "year_month":"2011-02", "year_month_day":"2011-02-21", "year_month_day_hour":"2011-02-21T19", "year_month_day_hour_min":"2011-02-21T19:00" }, "description":"Twenty-four hours a day ...", "disk_reference":"5576219606202173393", "duration":3600, "is_canonical":true, "is_live":true, "is_tv":true, "last_indexed":"2011-12-08T06:37:07Z", "matref":"beh6572", "pcrid":"crid://fp.bbc.co.uk/GEIXG", "pips_programme":{ "display_subtitle":"21/02/2011", "display_title":"BBC News", "episode_count":null, "formats":[ "bulletins" ], "genres":[ "news" ], "long_synopsis":"Twenty-four hours a day ...", "medium_synopsis":"Twenty-four hours a day ...", "pid":"b00yv5f6", "position":null, "programme_name":"21/02/2011", "series_number":null, "short_synopsis":"Twenty-four hours a day ..." }, "scrid":"crid://fp.bbc.co.uk/KCINMS", "snippets_genres":[ "news/bulletins", "news" ], "start":"2011-02-21T19:00:00Z", "title":"BBC News", "num_mentions":{ "libya":128 } }, ... ],}
Query parameters
| Parameter | Values | Description |
|---|---|---|
| keywords | Text | Text to query the programme subtitles |
| title | Text | Text to query the programme title |
| pid | Text | Comma separated list of PIPs pids |
| disk_reference | Text | Comma separated list of Redux disk references |
| genre | Text | Comma separated list of PIPs genres |
| channels | Text | Comma separated list of channel names, e.g. bbcone,bbctwo |
| citations | Integer | No. of citations to return, default 0, max 150 |
| limit | Integer | No. of results to return, default 10, max 50 |
| offset | Integer | Results number to start from, default 0 |
| order | “score”,”start”,”random” | Order by, default is score (relevance) |
| order_dir | “desc”,”asc” | Order direction, default is desc |
| start_date | ISO 8601 Date (ish) | Programmes must start after this date, e.g 2010-01-01T10:10:10+01:00 |
| end_date | ISO 8601 Date (ish) | Programmes must start after this date, e.g 2010-01-01T10:10:10+01:00 |
| filters | Filter Query | Advanced use, filter on a field, see recipes below for more info |
| facets | Facet Query | Advanced use, facet on a field(s), see recipes below for more info |
Recipes
Find all mentions of the word “Ferrari” in the series “Top Gear”
/snippets/search?keywords=ferrari&title="Top%20Gear"&api_key=aaabbbccc123123123
Show the citations for all mentions of the word “Ferrari” in the series “Top Gear”
/snippets/search?keywords=ferrari&title="Top%20Gear"&citations=150&api_key=aaabbbccc123123123
Find all “sport/motorsport” and “sport/cycling” programmes
/snippets/search?genre=sport/motorsport,sport/cycling&api_key=aaabbbccc123123123
Find all films broadcast Christmas Day 2010
/snippets/search?filters=pips_programme_formats:films&start_date=2010-12-25T00:00:00&end_date=2010-12-25T23:59:59&api_key=aaabbbccc123123123
Exclude repeats and live broadcasts from your results
/snippets/search?filters=-is_live:true&filters=is_canonical:true&api_key=aaabbbccc123123123
Exclude the “news” and “weather” genres
/snippets/search?filters=-snippets_genres:news,weather&api_key=aaabbbccc123123123
Query the PIPs synopsis
/snippets/search?filters=pips_programme_long_synopsis:hello&api_key=aaabbbccc123123123
Facet by genre
/snippets/search?facets=pips_programme_genres&api_key=aaabbbccc123123123{ facets: { pips_programme_genres: { childrens: 149098, childrens/activities: 9658, childrens/drama: 11485, childrens/entertainmentandcomedy: 109569, childrens/factual: 12419, ... } }}
Facet by month of the year
/snippets/search?facets=date_facet_month&keywords=christmas&api_key=aaabbbccc123123123{ facets: { date_facet_month: { 01: 4323, 02: 2233, 03: 2034, 04: 1835, 05: 1921, 06: 1703, 07: 2373, 08: 2564, 09: 2710, 10: 3610, 11: 4678, 12: 8941 } }}
Differences in subtitle quality
The BBC subtitles every TV show it broadcasts. Around 70% of the subtiles files are produced before the show is transmitted. These pre-recorded subtiles are very high quality - usually a verbatim transcription of everything that is said within a show. In some genres (e.g. Drama, Comedy) there is also a small amount of non-verbal description (e.g. “A GUNSHOT IS HEARD”, “LAUGHTER”). This is usually distinguisable by being in full caps. The timing information for pre-recorded subtitles is also very good, rarely off by more than a second or two.
The remaining 30% of shows were subtitled live. These shows have more errors because they were created by a human subtitler using speech-to-text technology to try and keep up with the live feed. As a result you’ll find that live subtitles have missing words, substituted words and other errors. In addition the timings are typically 5-10 seconds out of sync. Because of the way the live subtitles are transmitted we can flag them up and offer queries that exclude live shows (if required).
A note about canonical broadcasts
As off December 2011 there were 325,000 programmes in Redux. Of these, around 60% were repeats. Rather than give users multiple instances of the same show in their search results we’ve developed a system to determine the ‘canonical’ (i.e first broadcast) version of any show and only return canonical shows in search results. You can run queries to include repeats if you choose.
A note about the dataset
Redux has a copy of every programme broadcast on the London Transmitter since June 28th 2007. That means all of BBC1, BBC2, BBC3, BBC4, CBeebies, CBBC and BBC News for that period. Because it’s take from the London transmitter we don’t have any regional programming.
Missing data
We estimate that around 5,000 or 1.5% of the 325,000 Redux programmes have missing subtitles files (and therefore, by extrapolation, around 2000 of the canonical shows). This is due to a techical fault at the point of broadcast. We have decided to keep these shows in the database as it’s still possible to watch the show. What this means is there are 5000 shows in the database that will appear during open programme search (e.g. Programme = EastEnders) but which disappear if you start to search for keywords within that show (e.g. Programme = Eastenders, keyword = Queen Vic)
Rights
This API is currently only for use by BBC Staff and authorised third parties. The audio and video content comes under the 1988 Copyright Act and literary copyright comes into play for longer subtitle extracts. If you are planning to develop a public-facing product please contact Rob Cooper () for a run down of the current rights issues. Rest assured we are working with the BBC Rights Group to secure the rights to wider use of this metadata.
Contact Us
We welcome any feedback, bug and feature requests. Please contact Matt Haynes () or Rob Cooper ()