.. index:: populate postgres

.. _populatepostgres-chapter:

Populate PostgreSQL for the first time
======================================

Socorro supports multiple products, each of which may contain multiple versions.

* A product is a global product name, such as Firefox, Thunderbird, Fennec, etc.
* A version is a revision of a particular product, such as Firefox 3.6.6 or Firefox 3.6.5
* A branch is the indicator for the Gecko platform used in a Mozilla product / version. If your crash reporting project does not have a need for branch support, just enter “1.0” as the branch number for your product / version.

Customize CSV files
-------------------

Socorro comes with a set of CSV files you can customize and use to bootstrap
your database.

Set up environment
::
  make virtualenv
  . socorro-virtualenv/bin/activate
  export PYTHONPATH=.

Load the Socorro schema
::
  ./socorro/external/postgresql/setupdb_app.py --database_name=breakpad

Customize CSVs in tools/dataload/, at minimum you probably need to bump the dates and build IDs in
::
  raw_adu.csv reports.csv releases_raw.csv

You will probably want to change "WaterWolf" to your own
product name and version history, if you are setting this up for production.

See :ref:`databasetablesbysource-chapter` for a complete explanation
of each table.

Run backfill function to populate matviews
------------------------------------------
Socorro depends upon materialized views which run nightly, to display
graphs and show reports such as "Top Crash By Signature".

IMPORTANT NOTE - many reports use the reports_clean_done() stored
procedure to check that reports exist for the last UTC hour of the
day being processed, as a way to catch problems. If your crash
volume does not guarantee one crash per hour, you may want to modify
this function in socorro/external/postgresql/raw_sql/procs/reports_clean_done.sql
and reload the schema
::

  ./socorro/external/postgresql/setupdb_app.py --database_name=breakpad --dropdb

ALSO - the backfill procedure ignores any data over 30 days old.
Make sure you've adjusted the dates in the CSV files appropriately,
or change these functions in socorro/external/postgresql/raw_sql/procs/backfill_*.sql
and reload the schema as above.

Normally this is run for the previous day by cron_daily_matviews.sh
but you can run the following function to bootstrap the system
::

    psql breakpad -c "SELECT backfill_matviews()"

This is normally run by the import.sh, so take a look in there if
you need to make adjustments.

There also needs to be at least one featured version, which is
controlled by setting "featured_version" column to "true" for one
or more rows in the product_version table. The import script will go
ahead and set all imported versions as featured.

After modifying CSV files, use the import script to load the data
::
  ./tools/dataload/import.sh
