Google BigQuery

This article describes how to create the Google BigQuery collector, and includes:

• Creating a Google BigQuery data source
  
• Creating a Google BigQuery stream query

Creating a Google BigQuery data source

1. In the Navigation Panel, click Catalog.
2. Use the Search box OR click the Databases filter to locate the data source.
3. Hover over the Google BigQuery tile, and click Start. The Athena SQL dialog is displayed, as shown below.
   Note: If the data source has already been used, a dialog is displayed in which you can select from one of the listed sources. Alternatively, create a new source by clicking Add a new source.
   
   
4. Sign in with your Google account and allow anodot.com to access your Google account. The Stream Query window is displayed.

Creating a Google BigQuery stream query

If you have just created a BigQuery data source, skip to step 3.

1. In the Sources page (accessed by clicking INTEGRATIONS > Sources in the Navigation Panel), choose the BigQuery source for which you want to create a stream query. 
   Note: The streams associated with that source are displayed. If the Streams panel is empty, no stream queries exist for that source.
2. Hover over the Google Big Query data source, and click + New Stream. The Stream Query page is displayed.
   
   
3. Choose a project from the Choose Project dropdown menu. The list is searchable - just start typing the project name you are looking for. 
4. Choose a Data Set (optional).
5. Click the Query Schedule Edit icon to open the Query Schedule page.
   
6. In Query every, choose a query interval.
7. In Time Zone, choose a time zone according to which the records will be processed. The default is UTC.
8. In Collect Since, choose a time span of records to query while initializing the stream.
9. In Delay (Minutes), set the delay. The default is 5 minutes.
   Note To minimize partial query results, set the delay according to the delay your internal process takes to prepare the data records.
10. In Query Backfill Policy, set the backfill missing data records policy after stream initialization. The default is 0. 
    Examples
    0 = Query only last day/hour 
    1 = Query previous and last day/hour
    50 = Query previous 50 last days/hours
11. Click × to accept your query schedule and return to the Stream Query page.
12. Click Compose Query to open the Query Editor.
    
    
    
13. Use the sample text and guidelines to build a query.
14. To use Standard SQL, deselect the Use Legacy SQL. 
15. Click Preview Query.
16. Once you are satisfied with the query, click CONTINUE to return to the Stream Query page - the Dates, Measures and Dimensions  are displayed.
    
17. To edit any of the Measures & Dimensions, click Edit icon, to open the Measures & Dimensions page.
    
18. Edit the parameters you want to change:
    i.   Date
    ii.  Measures
    iii. Dimensions
19. Click × to accept the chosen parameters and return to the Stream Query page.
20. Click NEXT to display the Stream Table. 

See Also: 
Using Data Collectors
Collecting and Streaming Data
Stream Tables
Stream Summaries