Schools Database

A comprehensive system for managing and analyzing Australian school data. Import CSV files, run scraping tasks, and search or export results.

Getting Started

1. Request Access

To use the Schools Database, you need to be logged in as a superuser.

  • Click the "Login" link in the top navigation
  • If you don't have an account, contact the AFR datajournalism team
  • Once logged in, you'll have access to all features of the application

2. Import Schools Data

Start by importing school data from the Acara website.

  • Navigate to Import in the main menu
  • Select a year you want to import for. It should be a year that has not already been imported.
  • The system will import the current list of Australian schools from Acara.
  • Monitor the import progress in real-time using the task panel

3. Scrape School Data

After importing, run scraping tasks to fetch the latest school information.

  • Click on the year you want to scrape, from the table of imported years. The row should open up to reveal the scrape tools.
  • Run the scraping tasks for Profile, Finances, and NAPLAN pages
  • Monitor progress and view screenshots for each scraping step

4. Search and Export

Use the SQL Query Tool to analyze your processed data.

  • Navigate to Query Tool in the main menu
  • Write and execute SELECT queries against your data
  • View results in a paginated table format
  • Export query results to CSV for further analysis
  • Use the query history to reuse previous queries
Security & Performance Notes
  • Only SELECT queries are allowed in the Query Tool
  • Queries are limited to 45 seconds execution time and 100,000 rows
  • All data processing happens asynchronously in the background
  • Real-time progress updates are provided via WebSocket connections

Quick Links

Data Tables

The system manages the following main data entities:

  • Schools – Core table for school records and metadata (name, IDs, type, state, etc.)
  • Addresses – Physical addresses for each school (city, state, postal code, country, coordinates)
  • Import Events – Tracks each CSV import batch (year, timestamp, file, scrape options)
  • Import Queue – Queue of scraping tasks for each school in an import (status, progress, retries, notes)
  • School Profiles – Yearly profile data for each school (sector, type, staff, enrolments, student background, etc.)
  • School Finances – Yearly finances for each school (income, funding, capital expenditure, per-student amounts)
  • School NAPLAN – Yearly NAPLAN results for each school (scores, participation rates, national averages, error margins)

System Architecture

The Schools Database is built on Django with the following key components:

All data processing tasks run asynchronously, providing real-time progress updates through WebSocket connections. The system is designed for high performance and reliability when handling large datasets.


Version Information

App Version
2.1.0
Core Version
1.1.8
Git Commit
f9f17f7
Dirty State