Dev Team Portal

Appearance

Server Maintenance Template ​

Updated: 2025-11-12

pre update tasks: days before

  • Create updated EC2 AMIs + launch templates with latest patches (Windows updates etc)
  • Launch updated AMIs via Auto Scaling for Web, Web Services, Waiting Room
  • Update in-service instances (Scheduled Tasks, Staging, Web VPC RDP, osTicket)

pre update tasks: day before

  • Maintenance announcement on status page (Scheduled)
  • Maintenance announcement in TicketSource Messenger
  • CF: set environment variables COMPLETION_TIME_UK + COMPLETION_TIME_ET + COMPLETION_TIME_PT on "web" worker

update tasks: minutes before

  • API (PRINT SERVER): set blocking code in printServer_api_poll.php + printServer_api_template.php + printServer_api_link.php
  • EC2: set "Web-target-group" target group health check URL to /uptime-monitor-maintenance
  • EC2: set "Web-services-target-group" target group health check URL to /uptime-monitor-maintenance
  • SCHEDULED TASKS: (Run as administrator) D:\TicketSource-scheduled-tasks\task\disable-response-time-monitor-tasks.bat
  • CF: ticketsource.com > Security > Settings > Bot traffic > Allow verified bots => Off
  • CF: ticketsource.co.uk > Security > Settings > Bot traffic > Allow verified bots => Off
  • SLACK: Scheduled maintenance start

update tasks: start update

  • Log out of back office

  • STATUS: update maintenance announcement on status page (In Progress) + component status (Under Maintenance)

  • SCHEDULED TASKS: (Run as administrator) D:\TicketSource-scheduled-tasks\task\disable-all-scheduled-tasks.bat

  • SCHEDULED TASKS: stop instance

  • CF: set environment variable MAINTENANCE = 1 on "web" worker

  • CF: set environment variable MAINTENANCE = 1 on "web-services" worker

  • DB: disable firewall rule "SAP SQL Anywhere DB"

  • DB (use "dba" login):

    DELETE FROM DBA.patronSession;
DELETE FROM DBA.promoterPrintQueue;
DELETE FROM DBA.promoterSession;
DELETE FROM DBA.promoterUserAccountSession;
DELETE FROM DBA.seatingSeatReservationSession;
DELETE FROM DBA.useraccountSession;

  • DB:

    ALTER DATABASE SAVE CACHE;

update sub tasks

  • DB: stop DB services

  • DB: stop instance

  • DB: detach volumes (D: + E: + F:)

  • DB: remove termination protection

  • DB: destroy instance

  • DB: launch new AMI

  • DB: attach volume D:

  • DB: attach volume E:

  • DB: attach volume F:

  • DB: start DB service

  • DB (conditional): if DB service doesn't start => refresh the encrypted DB configuration file

    • copy local d:\ticketsource-local\db\live\ticketsource-db-configuration.txt to instance c:\ticketsource
    • command line:
      cd c:\ticketsource
dbfhide -wm ticketsource-db-configuration.txt ticketsource-db-configuration.enc
start DB service
delete c:\ticketsource\ticketsource-db-configuration.txt

  • DB (use "dba" login):

    ALTER DATABASE RESTORE CACHE;

  • DB: apply clean session database => Run as administrator C:\TicketSource\db-session-cleandb\initSessionDB.bat

  • DB (conditional): if DB SESSION service doesn't start => refresh the encrypted DB SESSION configuration file

    • copy local d:\ticketsource-local\db\live\ticketsource-session-db-configuration.txt to instance c:\ticketsource
    • command line:
      cd c:\ticketsource
dbfhide -wm ticketsource-session-db-configuration.txt ticketsource-session-db-configuration.enc
start DB SESSION service
delete c:\ticketsource\ticketsource-session-db-configuration.txt

  • DB BACKUP: Windows update

  • RDS: apply maintenance updates

  • API (PRINT SERVER):

    sudo apt update
sudo apt upgrade`
sudo service php7.0-fpm restart

  • AUTH:

    sudo apt update
sudo apt upgrade
sudo service php7.3-fpm restart

  • API:

    sudo apt update
sudo apt upgrade
sudo service php7.4-fpm restart

  • DB BACKUP: stop instance

  • DB: cache restored => enable firewall rule "SAP SQL Anywhere DB"

end of update tasks

  • SOURCETREE (optional): create new release

  • STAGING (optional): deploy

  • DB BACKUP: start instance + check db backup

  • SCHEDULED TASKS: start instance

  • TEST BOOKING

  • SCHEDULED TASKS: (Run as administrator) D:\TicketSource-scheduled-tasks\task\enable-all-scheduled-tasks.bat

  • EC2: set "Web-services-target-group" target group health check URL to /uptime-monitor

  • EC2: set "Web-target-group" target group health check URL to /uptime-monitor

  • CF: set environment variable MAINTENANCE = 0 on "web" worker

  • CF: set environment variable MAINTENANCE = 0 on "web-services" worker

  • CF: ticketsource.com > Security > Settings > Bot traffic > Allow verified bots => On

  • CF: ticketsource.co.uk > Security > Settings > Bot traffic > Allow verified bots => On

  • API (PRINT SERVER): unset blocking code in printServer_api_poll.php + printServer_api_template.php + printServer_api_link.php

  • STATUS: update maintenance announcement on status page (Verifying) + server status (Operational)

post update tasks

  • EC2: update CloudWatch dashboard: DB (c:) Volume
  • DB: ALTER DATABASE DROP CACHE;
  • LOGFLARE: check logs
  • SCHEDULED TASKS: (Run as administrator) D:\TicketSource-scheduled-tasks\task\enable-response-time-monitor-tasks.bat
  • STATUS: update maintenance announcement on status page (Completed)
  • SLACK: Scheduled maintenance end