Skip to main content

Data Release Process

Overview​

The publisher source of the underlying data (e.g. government agencies, banks, give-to-get SEO applications, data aggregators) makes data available on a given publication date. There is typically a lag between the last day the source measures data for an upcoming release and the day the source publishes the data. Cybersyn consistently monitors the source and updates the data product on Snowflake Marketplace. To go deeper, see how this process works for public data below.

screenshot

Public Data​

screenshot

For our public data sources, Cybersyn schedules the data updates based on a sensor. This sensor reads when new data is pushed to the source page (eg. FRED, BLS, Data Commons, StatCan, etc.).

_Please note: Cybersyn’s data releases are dependent on our public sources own release schedules; therefore, if there is a delay by the public source release, Cybersyn data tables will also have a delay in updates.

Once this sensor goes off, Cybersyn automates the data formatting and data cleansing processing.

Upon job completion, the data is updated in our Snowflake instance then replicated (shared) to every cloud region our clients host their instance of Cybersyn Data Product tables.

Data Replication Downtime​

A 10 minute downtime occurs when there is a schema migration or a new table is published. This is due to the time it takes to replicate the data across every cloud region that our clients host their instance of the Cybersyn Data Product tables.