The IEM attempts to maintain a high fidelity database of National Weather Service watch, warnings, and advisories (WaWA) that contain VTEC encoding. The implementation of VTEC started in October 2005 for some WaWA event types and at later dates for others. For Flash Flood Warnings, the VTEC implementation was in 2007. VTEC made programmatic parsing of the events much simpler and even tenable. For events prior to VTEC implementation, there is great gnashing of teeth.
So without VTEC, one has to do a bunch of heuristics attempting to figure out which counties/zones were covered by a given event and what the expiration time was for the event. At one point many moons ago, the IEM got a dump of simple warning metadata back to 1986 that covered pre-VTEC warnings like Flash Flood Warnings. This metadata was rather bare bones and did not contain things like the raw text that went along with the warning.
Since the IEM's database is about the only known game in town in this space, a number of research groups have been looking more closely at the Flash Flood Warning database for dates prior to 2007. I took some time attempting to clean up the period between 2002 to 2007 and wish to document here some changes made in this space:
- An attempt, if mostly in vain, was made to merge up the issuance FFW raw text file and any FFS updates with the raw metadata mentioned above. The result is that most events, for example DMX #1 for 2006, there is now issuance text and some other ancillary metadata.
- One of the issues driving all this, is the usage of Flash Flood Emergencies. The IEM has an accounting that received some quality control for this period due to the work in this space.
- This gets way off into the weeds, but assigning a canonical issuance time to a NWS text product is fraught with pain. Work was done to rectify products during this period to use a presently used methodology which considers the mass news disseminator (MND) header as gospel. So some text products available within the AFOS interface had small changes to their URIs.
- Any polygons that came with the FFWs or FFSs came along for this ride and got entered into the database. Previously, there was very few of these for the non-official implementation of storm based warnings.
- The yearly GIS shapefiles for this period got regenerated.
Woof, that's a lot of confusing garble above. You may be wondering why this did not cover dates prior to 2002. Well the pain only gets worse for years prior due to very adhoc text and too many quirks to account for. Congratulations if you learned something reading this and please let me know of any questions or problems you have with this dataset!