Comment on page
12/5/23: Financial & Economic Essentials - Added interest rates posted by major chartered banks in Canada for selected products; Added secured overnight financing rates (SOFR) and US Treasury bill rates
Added interest rates for selected products posted by the six major chartered banks in Canada to the
FINANCIAL_FRED_VARIABLE_SERIES_ID_CROSSWALKtables. The typical rate is calculated based on the statistical mode of the rates published by the six banks.
The posted rates cover:
- Prime lending rate
- Conventional mortgages
- Guaranteed investment certificates
- Personal daily interest savings
- Non-checkable savings deposits
Added secured over night financing rates (SOFR) and US Treasury bill rates to the
Series added (FRED ID):
- Secured Overnight Financing Rate (SOFR)
- 30-Day Average Secured Overnight Financing Rate (SOFR30DAYAVG)
- 90-Day Average Secured Overnight Financing Rate (SOFR90DAYAVG)
- 180-Day Average Secured Overnight Financing Rate (SOFR180DAYAVG)
- Secured Overnight Financing Rate Index (SOFRINDEX)
- 3-Month Treasury Bill Minus Federal Funds Rate (TB3SMFFM)
- 6-Month Treasury Bill Minus Federal Funds Rate (TB6SMFFM)
12/5/23: Canada Government Essentials - Added interest rates posted by major charted banks in Canada for selected products
Added interest rates for selected products posted by the six major chartered banks in Canada to the
CANADA_STATCAN_TIMESERIEStables. The typical rate is calculated based on the statistical mode of the rates published by the six banks.
The posted rates cover:
- Prime lending rate
- Conventional mortgages
- Guaranteed investment certificates
- Personal daily interest savings
- Non-checkable savings deposits
12/4/23: Financial & Economic Essentials - Added Monthly State Retail Sales (MSRS) from the US Census Bureau
Added monthly state-level retail sales by NAICS codes to the
FINANCIAL_FRED_VARIABLE_SERIES_ID_CROSSWALKtables. The US Census Bureau publishes year-over-year percent changes for each state using a composite model of Monthly Retail Trade Survey (MRTS) data, administrative data, and third-party data beginning in January, 2019.
Year-over-year percent change estimates for the following industries are now available at the state level:
- Total Retail Sales Excluding Nonstore Retailers
- Building Material and Garden Equipment and Supplies Dealers
- Clothing and Clothing Accessories Stores
- Electronics and Appliance Stores
- Food and Beverage Stores
- Furniture and Home Furnishings Stores
- Gasoline Stations
- Health and Personal Care Stores
- Motor Vehicle and Parts Dealers
- Sporting Good, Hobby, Musical Instrument and Book Stores
- Miscellaneous Stores
11/30/23: Government Essentials - Added global trade, tariff, and import relationship data from the World Trade Organization (WTO)
Added global trade flows, imposed tariffs, and trade interactions between countries from the World Trade Organization (WTO) . The data details export and import figures for goods and services across different countries and regions, tariff rates and structures that WTO member countries apply to imports from other nations, trade dependencies between countries, and the balance of trade between specific pairs of nations.
WORLD_TRADE_ORGANIZATION_ATTRIBUTEStable details the global trade, tariff, and import relationship statistics tracked by the World Trade Organization (WTO).
WORLD_TRADE_ORGANIZATION_TIMESERIEStable provides timeseries values by date for the reported trade statistics by country, country group, and global region (as defined by the World Trade Organization).
11/30/23: Government Essentials - Added global health indicators from the World Health Organization (WHO)
Added 1,100+ health-related indicators for 194 members of the World Health Organization (WHO) and their associated country groups and global regions. Example metrics include alcohol consumption among adolescents and adults, tobacco control policies, abortion rates, accessibility of dementia care services, and adolescent fertility rates. Environmental health indicators including air pollution's impact on mortality rates and disability-adjusted life years (DALYs) as well as deaths attributable to the environment are also included.
WORLD_HEALTH_ORGANIZATION_ATTRIBUTEStable details the health statistics tracked by the World Health Organization (WHO).
WORLD_HEALTH_ORGANIZATION_TIMESERIEStable provides timeseries values by date for the reported health indicators by country, country group, or global region (as defined by global organizations like UNICEF, the United Nations, the World Bank, and the World Health Organization).
11/30/23: Government Essentials, Financial & Economic Essentials - Added country groups and regions from the WHO, WTO, and UN to geography tables
Added additional country groups and geography types to the
GEOGRAPHY_INDEXfrom the World Health Organization (WHO), World Trade Organization (WTO), and United Nations (UN). The member countries of the added geographies are mapped in the
GEOGRAPHY_RELATIONSHIPStable. Select new geographic regions include:
- BRICS members
- World Trade Organization (WTO) members
- Association of Southeast Asian Nations (ASEAN)
- UNICEF regions
- United Nations regions
- United Nations Sustainable Development Goal (SDG) regions
- World Bank regions
- World Health Organization (WHO) regions and income regions
- World Bank regions and income groups
Added US Export Sales Reporting (ESR) data on weekly export sales activity for 40+ US agricultural commodities sold abroad from the US Department of Agriculture's (USDA) Foreign Agricultural Service (FAS).
US_DEPARTMENT_OF_AGRICULTURE_COMMODITIES_ATTRIBUTESnow includes the export of commodities in addition to the existing production, supply, and distribution variables.
US_DEPARTMENT_OF_AGRICULTURE_COMMODITIES_TIMESERIEStable provides the reported metrics for each commodity by
11/22/23: Financial & Economic Essentials - Added data from the Bank for International Settlements (BIS) on global banking conditions, property prices, and consumer price indicators
Added data from the Bank for International Settlements (BIS) on global banking conditions, property prices, and consumer price indicators
The Bank for International Settlements (BIS) is an international financial institution owned by 60+ central banks that represent countries accounting for ~95% of global GDP. As part of its mission is to support international monetary and financial cooperation, the BIS acts as a bank for central banks across the world.
Added the below BIS data to two new tables,
BANK_FOR_INTERNATIONAL_SETTLEMENTS_ATTRIBUTES, which describes the metrics tracked by BIS, and
BANK_FOR_INTERNATIONAL_SETTLEMENTS_TIMESERIES, which provides the values of metrics:
- Residential property prices
- Central bank policy rates
- Consumer price indicators
- Assets and liabilities of internationally active banks, including credit to the non-financial sector as well as the geographical and currency composition of a bank’s balance sheet
11/22/23: Consumer Spending Foundation - Added spend by geography for states and top 100 CBSAs. Added online/offline spend breakdown
Expanded the granularity of values in the
CONSUMER_SPENDING_TIMESERIEStable to include breakdown by geography. Geographies are grouped both by purchaser primary location (i.e. where the spender lives) and purchase location (i.e. where the physical store is located). The data covers US states and the top 100 core-based statistical areas (“CBSAs”) as measured by population.
- Purchaser geographies are represented in the
- Purchase location geographies are represented in
PURCHASER_PRIMARY_GEO_IDcan be used to join the data with Cybersyn’s geography tables such as the
GEOGRAPHY_INDEXbased on the
Added purchase channels covering online and offline spend.
- Purchase channels are reflected in the
CHANNELfield in the
CONSUMER_SPENDING_ATTRIBUTEStable and included in the
VARIABLE_NAMEvalues in the
Note that data for “purchase location” geographies only includes offline spend.
11/20/23: Consumer Spending Foundation - Added aggregations by NAICS codes and by 4-5-4 retail months
CONSUMER_SPENDING_ATTRIBUTEStables to include aggregations by NAICS (North American Industry Classification System) and 4-5-4 retail calendar months.
MART_VARIABLEfields in the the
CONSUMER_SPENDING_ATTRIBUTEStable and replaced them with
AGGREGATION_VALUE.The newly added fields can be used to filter to a desired level of aggregation including to the NAICS, MARTS Segment, and MCC levels.
Additionally, the following fields were added in anticipation of upcoming dataset expansions. Note that these fields only contain total values for now. Future iterations of the product will include more granular cuts of data.
11/20/23: Financial & Economic Essentials - Added 10 timeseries from FRED covering mortgage rates and additional CPI measures
Added 10 additional timeseries from FRED to the
- MORTGAGE15US: 15-Year Fixed Rate Mortgage Average in the United States
- MORTGAGE30US: 30-Year Fixed Rate Mortgage Average in the United States
- CUSR0000SEFV: Consumer Price Index for All Urban Consumers: Food Away from Home in U.S. City Average, Seasonally Adjusted
- CUUR0000SEFV: Consumer Price Index for All Urban Consumers: Food Away from Home in U.S. City Average, Not Seasonally Adjusted
- CUSR0000SETG01: Consumer Price Index for All Urban Consumers: Airline Fares in U.S. City Average, Seasonally Adjusted
- CUUR0000SETG01: Consumer Price Index for All Urban Consumers: Airline Fares in U.S. City Average, Not Seasonally Adjusted
- CUSR0000SS62031: Consumer Price Index for All Urban Consumers: Admission to Movies, Theaters, and Concerts in U.S. City Average, Seasonally Adjusted
- CUUR0000SS62031: Consumer Price Index for All Urban Consumers: Admission to Movies, Theaters, and Concerts in U.S. City Average, Not Seasonally Adjusted
- CUSR0000SEHB: Consumer Price Index for All Urban Consumers: Lodging Away from Home in U.S. City Average, Seasonally Adjusted
- CUUR0000SEHB: Consumer Price Index for All Urban Consumers: Lodging Away from Home in U.S. City Average, Not Seasonally Adjusted
11/20/23: Government Essentials - Expanded American Community Survey (ACS) history for 1,400+ population variables since 2005 for ~500K geographies
Added historical data from the American Community Survey (ACS) to the
AMERICAN_COMMUNITY_SURVEY_TIMESERIEStables for over 1,400 population variables dating back to 2005 at the following geographic entity levels: country, states, counties, cities, zip codes, core-based statistical areas (CBSAs), census tracts, and census block groups. Example population variable additions include age, race, income, employment status, immigration status, and household status.
Data is as up to date as the latest ACS publication.
11/16/23: Weather & Environmental Essentials - Added 6 tables covering disaster declaration and National Flood Insurance Program (NFIP) insurance data from FEMA
Added Federal Emergency Management Agency (FEMA) data on federally-declared disasters in the United States, disaster recovery public programs, and the National Flood Insurance Program (NFIP) insurance policies and claims. Six new tables were included in the release:
FEMA_DISASTER_DECLARATION_INDEX- Details for each federally-declared disaster (e.g. name, type, date, public assistance funding amounts). Table is unique by
FEMA_DISASTER_DECLARATION_AREAS_INDEX- Geographic entities (e.g. counties, cities) impacted by each federally-declared disaster. A disaster declaration can include multiple geographic locations.
FEMA_MISSION_ASSIGNMENT_INDEX- Work orders issued by FEMA since 2012 to other government agencies, supporting emergency response activation across the US (e.g. requesting transportation support from the US Department of Transportation during a hurricane).
FEMA_NATIONAL_FLOOD_INSURANCE_PROGRAM_CLAIM_INDEX- Details on National Flood Insurance Program (NFIP) claims including features of the insured property, information on the flood event precipitating the claim, the cost of the damage, and subsequent insurance payout amounts.
FEMA_NATIONAL_FLOOD_INSURANCE_PROGRAM_POLICY_INDEX- Details on National Flood Insurance Program (NFIP) policies including features of the insured property, building and contents insurance coverage, deductibles, rates, and policy durations.
FEMA_REGION_INDEX- Human readable names and details for each FEMA Region
11/13/23: Canada Government Essentials - Added 6 timeseries from Statistics Canada covering Canadian debt securities and real estate development
Added 6 new timeseries to the
CANADA_STATCAN_ATTRIBUTEStables from 3 of Statistics Canada’s underlying sources:
- Bank of Canada: Government of Canada debt gross new issues, retirements and net new issues, and par values
- Annual Survey of Service Industries: Real estate agents, brokers, and appraisers summary statistics (e.g. salaries, wages, operating revenue)
- Canada Mortgage and Housing Corporation:
- Absorptions and unabsorbed inventory, newly completed dwellings, by type of dwelling unit in select census metropolitan areas
- Absorptions and unabsorbed inventory, newly completed dwellings, by type of dwelling unit in census agglomerations of 50K+
- Housing starts, under construction and completions in select census metropolitan areas
- Housing starts, under construction and completions in census agglomerations of 50K+
- American Community Survey
- Canada Statcan
- Carbon Credit Purchases
- Carbon Intensity
- Company Index and Characteristics
- Data Commons
- Domain Index
- Geography Characteristics
- Home Mortgage Disclosures
- NOAA Weather Stations and Metrics
- OpenFIGI Security Index
- PermID Security Index
- Points of Interest (POIs)
- Urban Crime
- US Addresses
- US Treasury
- USPS Address Changes
Expanded values in the
CONSUMER_SPENDING_TIMESERIEStable to include nominal estimates. New measures include estimates for revenue ($), transactions (#), and average order value ($). These newly added variables build on the existing year-over-year (%) estimates for those measures already in the table.
Nominal values can be used to measure market share or compare average transaction amounts across retailers. Because these estimates are based on a panel of consumer spending, they are not meant to be projections of the entirety of US spending but they are accurate as a measure of relative spend. For example, the sum of all Chipotle spend will not equal the company's actual spend but Chipotle's market share relative to McDonald's should be correct.
The newly-added variables in the timeseries table include matching variables in the
CONSUMER_SPENDING_ATTRIBUTEStable with the new variables being
CALENDAR_INDEXtable which compiles common calendars into a single table. Each calendar type has a unique
CALENDAR_ID, which allows users to select which calendar type they want to use. Individual periods within each calendar type include period start and end dates.
CALENDAR_INDEXcurrently includes regular calendar periods (days, weeks, months, quarters, and years) and 4-5-4 retail calendar periods (4-5-4 retail months, quarters, and years).
The 4-5-4 retail calendar is a standardized accounting and reporting calendar system used by many retailers, where each fiscal year is divided into 13 weeks, aiming to align with seasonal variations and facilitate more accurate financial comparisons.
11/3/23: Government Essentials - Added global agricultural commodity production and distribution data from the USDA
Added two tables sourced from the US Department of Agriculture's (USDA) Foreign Agricultural Service (FAS) which provides production, supply, and distribution data on agricultural commodities for both the United States and other producing and consuming countries since 1960.
us_department_of_agriculture_commodities_attributesdescribes the production, supply, and distribution metrics tracked for each commodity by the USDA.
us_department_of_agriculture_commodities_timeseriestable provides the reported metrics for each commodity and country.
10/25/23 - Financial & Economic Essentials - Added detailed branch-level data and Summary of Deposits (SOD) data from the FDIC
The Summary of Deposits (SOD) data from the FDIC is an annual survey, capturing branch-level deposits as of June 30 for all FDIC-insured institutions, including U.S. branches of foreign banks.
fdic_branch_locations_indextable provides details on FDIC-insured bank branches, including branch-specific location information as well as institution-level regulatory and insurance data.
fdic_summary_of_deposits_attributestable describes the deposit types tracked by the Summary of Deposits (SOD) survey.
fdic_summary_of_deposits_timeseriestable provides the results of the annual Summary of Deposits (SOD) survey going back to 1994 for banks’ branch-level deposits.
10/19/2023: Government Essentials - Added US Federal Government Revenue Collections from the US Treasury Fiscal Data
The US Treasury provides a daily overview of net federal revenue collections from income tax deposits, customs duties, fees for government services, fines, and loan repayments. These collections undergo electronic and/or non-electronic processing, involving various channels such as mail, internet, banking, and over-the-counter transactions, all of which are comprehensively incorporated within this dataset.
us_treasury_revenue_collections_timeseriestable provides daily net collections amounts broken down by tax category and processing channel. The
us_treasury_revenue_collections_attributestable details each collection method reported by the US Treasury.
10/18/23: US Insurance & Healthcare Provider Foundation - Added Form 5500 Schedule A Part 1 insurance data from the US Department of Labor.
Expanded the US Department of Labor data to include information found on Form 5500 Schedule A Part 1. The new table,
us_department_of_labor_form_5500_broker_index, provides commission and fee amounts received by a broker for an insurance policy. Additional information about the brokers in the table includes their address, classification as an insurance broker, as well as notes pertaining to the compensation disbursed to them.
us_department_of_labor_form_5500_broker_indexcan be joined to insurance carrier and policy information to individual Form 5500 filings, using
10/16/23: Financial & Economic Essentials - Added exchange rates from the Bank for International Settlements (BIS)
Due to the discontinuation of certain currency conversion pairs by the European Central Bank (ECB), our primary source for daily FX rates, we have sourced a number of these pairs from an alternative source, the Bank for International Settlements (BIS), for ongoing history. The Bank for International Settlements will be used to get data for the following currency conversion pairs after September 27, 2023: USD:AED, USD:ARS, USD:CLP, USD:COP, USD:DZD, USD:MAD, USD:PEN, USD:QAR, USD:SAR, USD:TWD, USD:UAH.
Full history from the Bank for International Settlements was added for the following currency pairs: USD:ALL, USD:AUD, USD:BAM, USD:BHD, USD:BND, USD:EUR, USD:GBP, USD:IRR, USD:ISK, USD:KWD, USD:KZT, USD:LKR, USD:MKD, USD:MUR, USD:NPR, USD:NZD, USD:OMR, USD:RSD, USD:RUB, USD:TND, USD:TTD, USD:UYU, USD:VEF, USD:XDR.
10/11/23: Government Essentials - Added population variables to the American Community Survey tables.
american_community_survey_timeseriestables to include additional population variables related to income, age, and educational attainment.
New series include Household Income in the Past 12 Months (Inflation-Adjusted), Educational Attainment for the Population 25 Years and Over, and Age of Householder By Household Income in the Past 12 Months (Inflation-Adjusted). These series are available by multiple breakdowns (ex. income, age, gender, etc.).
Added year-over-year estimates for company-level spend, transaction, and AOV to the
CONSUMER_SPENDING_TIMESERIES. Company-level information is identified with the company_id field.
The company_id is a unique identifier assigned by Cybersyn to each company and is joinable to the
COMPANY_INDEX, which provides company names and other helpful identifiers such as CIK, LEI, PermID, and more. Note that when the company_id is null, then the row represents data for all companies.
geography_indextable to include U.S. Census Regions and Divisions.
geography_hierarchytable to include the relationships between U.S. Census regions and U.S. Census divisions; U.S. Census regions and U.S. states; and U.S. Census divisions and U.S. states.
Census regions include the United States Northeast, Midwest, etc. and census divisions include the United States Middle Atlantic, East North Central, etc.
- Added one new table to our dataset,
PERMID_SECURITY_INDEX, which includes security identifiers from Refinitiv’s PermID database. These are persistent identifiers for active and inactive securities across global asset classes. The table includes over 15K PermIDs for various securities.
- This data can be merged to the 13F filings data (
SEC_HOLDING_FILING_ATTRIBUTES) and can be mapped back to companies using the
10/06/23: Tech & Innovation Essentials - Added in repository of web domains plus included GitHub Archive & US Patents tables
9/18/23: Financial & Economic Essentials - Added 6 timeseries from FRED covering core CPI and industry and commodity-specific PPI data
Added 6 new timeseries from FRED to the
- PPIFIS: Producer Price Index by Commodity: Final Demand
- PCU4841214841212: Producer Price Index by Industry: General Freight Trucking, Long-Distance Truckload
- PCU4841224841221: Producer Price Index by Industry: General Freight Trucking, Long-Distance Less Than Truckload
- PCU3313133131: Producer Price Index by Industry: Alumina and Aluminum Production and Processing
- WPU101707: Producer Price Index by Commodity: Metals & Metal Products: Cold Rolled Steel Sheet and Strip
- CPILFESL: (CORE CPI) Consumer Price Index for All Urban Consumers: All Items Less Food & Energy in U.S. City Average
9/17/23: SEC Filings, LLM Training Essentials - Added
COMPANY_SECURITY_RELATIONSHIPtables; added PermIds
Added three new tables:
COMPANY_INDEXtable aggregates commonly used company identifiers (i.e. CIKs, EINs, and LEIs) into a single a single
company_id, which can be used across Cybersyn’s datasets as a unique identifier for corporate entities.
COMPANY_SECURITY_RELATIONSHIPtable maps OpenFIGI and PermID securities (i.e. securities with multiple "levels" such as OpenFIGI FIGI ID and OpenFIGI Share Class ID) to the Company.
COMPANY_CHARACTERISTICStable includes categorical characteristics of a Company (e.g. industry, address, previous names). A characteristic may be temporal with start and end dates indicating the range for which the data is valid.
9/15/23: Weather & Environmental Essentials - Added daily weather data from 80K+ weather stations across 180 countries; updated product name
New tables added:
NOAA_WEATHER_STATION_INDEXcontains metadata on the weather stations from the Global Historical Climatology Network daily (GHCNd) database, including mappings to Cybersyn’s
GEO_IDat the country-, state- and zip-level (where applicable).
NOAA_WEATHER_METRICS_ATTRIBUTESincludes the daily weather variables tracked globally and their measurement details.
NOAA_WEATHER_METRICS_TIMESERIESprovides the details of the daily global weather variables recorded at each weather station.
Updated product name from "Emissions & Environment Essentials" to "Weather & Environmental Essentials"
9/15/23: US Insurance & Healthcare Provider Foundation - Added healthcare provider emails; combined
TELEPHONE_EXTENSIONinto one field; changed
TELEPHONEto array to accommodate numerous values
nppes_provider_addressestable with provider emails per address.
nppes_provider_addressestable into a single field,
TELEPHONE, and removed the
- Aggregated all values for
nppes_provider_addressestable are now uniquely defined by NPI and full address.
9/10/23: US Insurance & Healthcare Provider Foundation - Added table to relate taxonomy classifications to practitioners’ license numbers
NPPES_PROVIDER_TAXONOMY_AND_LICENSE_NUMBERStable that relates taxonomy classifications to practitioners’ license numbers. This table provides users the ability to filter for license numbers based on practitioners’ primary taxonomy.
9/7/23: SEC Filings - Added 13F filings to include data on quarterly investment fund managers’ holdings; added a securities index table based on OpenFIGI data
Expanded our dataset to include individual filings from 13F fund holding reports, which disclose the equity holdings of institutional investment managers. Added three new tables,
SEC_HOLDING_FILING_INDEXtable contains metadata from individual 13F filings including filing date and filing organization.
SEC_HOLDING_FILING_ATTRIBUTEStable includes securities' names, market value, number of shares held, and OpenFIGI IDs, which facilitate easier mapping and analysis to outside data sources. Table
OPENFIGI_SECURITY_INDEXcontains an index of over 2M securities listed on OpenFIGI and can be joined with table
TOP_LEVEL_OPENFIGI_ID- the unique identifier for each security in the two tables.
9/6/23: US Insurance & Healthcare Provider Foundation - Added contact information and broker payments to Form 5500 data
- Added new fields to
us_department_of_labor_form_5500_filing_indexwith Form 5500 contact information including name and phone numbers:
- Added new fields to
us_department_of_labor_form_5500_policy_indexwith payments to agents and brokers:
- Removed ~10k rows from
us_department_of_labor_form_5500_policy_indexthat had NULL values for each field as they filed no data around insurance policies from Form 5500 Schedule A.
8/31/23: US Insurance & Healthcare Provider Foundation - Added deactivated NPI numbers to NPPES data
Added a new table,
nppes_npi_index, that contains information on when NPIs were first issued, deactivated, or reactivated - dating back to 2005. This table also includes a boolean flag to indicate if an NPI is currently active.
Note that while all NPIs appear in the
nppes_npi_indextable, only actively registered NPIs as well as NPIs deactivated after August 1, 2023 appear in the
nppes_organization_attributestables. This means the dataset does not include attribute-level data (names, type of providers, specialization) on providers with NPIs deactivated before August 2023.
8/27/23: US Points of Interest & Addresses, US Housing & Real Estate Essentials - Added points of interest data from Overture Maps Foundation
point_of_interest_indextable, which includes names and categories for points of interest in the US. Each POI is uniquely identified by a
To tie POIs to addresses, we added a new column,
ADDRESS_ID, to the
us_addressestable to uniquely identify each individual address. This column allows users to join addresses to POIs using the new
ADDRESS_IDas the join keys for the
8/27/23: US Points of Interest & Addresses, US Housing & Real Estate Essentials - Added 7.2M new addresses, removed 49.8M duplicate addresses, deleted 1.2M addresses with
Added 7.2M new addresses covering points of interest from Overture Maps Foundation to the
Removed 49.8M addresses that were duplicative aside from minor variability in coordinates. Removed 1.2M rows from rows from the
us_addressestable where the
STREETvalue contained a string with value
Expanded our coverage of SEC documents to include the full text of 8-K filings and associated exhibits. 8-K filings include company press releases, earnings releases, and other major corporate events.
Added the full text of exhibits for 10-K and 10-Q filings. Exhibit types include lists of subsidiaries, merger agreements, and material changes in financial conditions. Exhibits are denoted in the
10-K EX-21 Filing Text).
sec_document_idcolumn. This field is a combination of the ADSH (accession number) and the document type (e.g. 10-K). This serves as a unique identifier for each individual component that makes up a filing in cases when one or more exhibits are included in a filing.
8/11/23: Government Essentials, US Housing & Real Estate Essentials, US Addresses & Geographic Areas - Added geospatial boundaries data for territories in the US and Canada
The Census Bureau and Statistics Canada publish geospatial boundaries data for their territories at multiple geographic levels. We added a table
geography_characteristicswith the boundary coordinates from the most recent releases in both WKT and GeoJSON formats. The table is joinable at different levels using Cybersyn's
GEO_IDis compatible with all Cybersyn listings that have geographic identifiers. Currently, the geographic levels covered include:
- State (US and Canada)
- County (US only)
- Census Tract (US only)
- ZIP Code (US only)
- Dissemination Area and Aggregate Dissemination Area (Canada only)
- Census Division and Census Subdivision (Canada only)
- Census Agglomeration and Census Agglomeration Part (Canada only)
- Census Metropolitan Division and Census Metropolitan Division Part (Canada only)
8/10/23: Financial & Economic Essentials - Added crosswalk to FRED series IDs & 107 new series from GDP, Employment Situation, Housing Starts, and Residential Construction reports
Added a new table,
financial_fred_variable_series_id_crosswalk, that enables a join between Cybersyn’s variable and FRED’s unique series ID
Added new series from the following four reports:
- Gross Domestic Product (data produced by the US Bureau of Economic Analysis)
- Employment Situation (US Bureau of Labor Statistics)
- New Residential Construction (US Department of Housing and Urban Development)
- Quarterly Starts and Completions by Purpose and Design (US Department of Housing and Urban Development)
The US government publishes contract opportunities and proposals to do business with the federal government via the System for Award Management (sam.gov) for contracts and awards with a value of at least $25,000. The data goes back to January 2002 and includes metadata providing descriptions of government contracts and the corresponding awards granted for those contracts.
7/31/23: Financial & Economic Essentials - Added
release_sourcefor better discoverability
release_name: The collection, group of data, or report from which a time series originates. This column can be used as a filter to find related series.
release_source: The organization (e.g. FDIC, Federal Reserve) that FRED collects the data from.
Two columns were added to
financial_fred_attributesto provide better categorization and discoverability:
- New datasets:
- Household income and finances: Household income and consumption, household credit liabilities, and household savings rates
- Prices and output: Core consumer price index, gross domestic product by industry group, and new housing price indices
- Updated datasets:
- StatCan archived a number of retail trade series and published new series. Old series are marked “Archived…” in the
reportfield. The latest series have been included to replace these.
- Schema changes:
- The following columns in the
canada_statcan_attributesview were updated. The deprecated columns will be removed on 7/28/2023.
age_groupwill be folded into
demographic_group, which applies more broadly to age ranges, income groups, and household makeups (e.g., 35 to 44 years, lowest income quintile, elderly persons not in an economic family)
measurewill be renamed
reportcolumn displays the StatCan dataset from which the data originates (e.g. Consumer Price Index, New Housing Price Index)
labour_force_statisticwill be renamed
statistic. The new statistic column will provide the label for the specific economic metric that is being reported (e.g. Median After Tax Income, Number of families)
Added information from US Department of Labor Form 5500 filings about company benefit providers and the insurance/benefit plans they offer. New tables include
Added Consumer Price Index (CPI), Average Prices (AP), Job Openings and Labor Turnover Survey (JOLTS), State and Metro Area Employment , Hours, & Earnings (SAE), Local Area Unemployment Statistics (LAUS) from the Bureau of Labor Statistics.
- Using USPS address change data, we added 3,000 zip codes (mostly PO Box) to the
- Using both the USPS address change and US Census Bureau data, we increased the coverage in
geography_relationshipstable with 6,500 new zip and city relationships. We now map 86% of zip codes to a city.
5/19/23: Government Essentials - Updated product name from Cybersyn Data Commons to Cybersyn Government Essentials
Rebranded Cybersyn Data Commons as Cybersyn Government Essentials. We updated the naming conventions for schemas, tables, and column names to make them consistent across all of Cybersyn’s existing and future data products.
Cybersyn will continue to support and update your older version of the Data Commons tables.
5/19/23: US Addresses & Geographic Areas, GitHub Events - Added source data from National Address Database (NAD)
Added the National Address Database (NAD) as a source to increase our US address coverage:
- Increased the coverage from 140 million addresses to more than 188 million.
- There is now at least one address in more than 85% of zip codes, up from 74% previously.
- Increased the portion of cities that are mapped to distinct IDs joinable to our other data sets from 24% to over 77%
Last modified 3d ago