There are an ever-growing number of technologies used to process data but which one to use for your project idea?
Guidance for Technology Use
While participants in the datathon are free to choose their own technology (so long as it is their own code and/or third party software that is appropriately licenced as open source), to implement their project, the golden rule is that the outcome must utilise at least two open datasets. The output of your efforts can be software, an infographic, a spreadsheet, or other graphical visualisation that presents your ambition in a suitable way.
If your entry is software-based, the ultimate outcome of should be a deployable asset with a web front end. The main engine of the development can be a processing powerhouse with an API, as long as there is some web façade that enables a casual user to inspect and appreciate the offering. Whatever technology is chosen, full details of it’s deployment requirements should be provided therewith. If the product is a containerised web application (e.g. WAR file) then the precompiled package should be provided. Where a project uses an API, this should be described in the README file. Where a project uses a downloaded dataset, this should be included in a locally accessibly resources folder so that the project can execute on any machine.
Data processing is a very active area. A myriad of tools and technologies exist to aid reporting and analytics of significantly large datasets. Lighweight tools such as scripting languages afford swift development opportunities but may lack some of the heavy-lifting capabilities of other technologies. Software environments such as R, offer free and powerful solutions to processing statistical data. No matter what technology choice, there will possibly exist a number of libraries (or add-ons) specifically designed to help with the task of data processing. Those submitting infographic, spreadsheet, or other non-software entries should clearly identify the datasets used and provide the supporting work files that enabled them to arrive at their final product.