Visual Arts Press Blog

The all knowing, ever present VAP knowledge base

In case of emergency, break glass…

Symptom: Website seems unresponsive or a known page is throwing a 500 Internal Server Error
Prescription: SSH into production and run $ top. If CPU is spiking (consistently greater than 85%) then something is wrong. The general culprits are either MySQL or Apache. Generally you can tell which is the problem from top by looking at CPU usage. Regardless you’ll probably want to do the following:

  • Verify MySql is operational and accessible by logging in: $ mysql -uroot -psvaW3bpr0ject
  • The site uses the sva_sf2 database, so in addition you can do: > use sva_sf2; select count(*) from pages; and you should get something greater than 7,000. In the off chance you should need to restore or compare against a previous database, daily backups are in /var/www/database_backups
  • If it seems that apache is borking (high CPU): $ sudo service httpd restart
  • If that still didn’t do it, then $ cd /var/www/html/sva/releases/release/ and run $ sudo ./clear_cache.sh, and restart Apache after that
  • If that fails and you think the problem is with mysql then first shutdown apache:
    • $ sudo service httpd stop
    • $ mysqladmin -uroot -psvaW3bpr0ject shutdown
    • $ sudo mysqld_safe
    • $ sudo service httpd start
  • If you cannot SSH into the box then there’s nothing you can do except get in touch with IT (Ian, Brian, Cosmin) and have them reach out to Rackspace to restart the server.
Symptom: CE course is not on the website/has the wrong info
Prescription: 90% of the time this is a problem with how the course was entered by the Registrar. The entire CE workflow is documented here (TL;DR) but the basic things you want to check are:
  • Make sure the course is in the Regbook database (connection details below)
  • All CE courses are stored in either one of two tables: regbook_sections_ft for full-time courses and regbook_sections_ce for regular non-credit CE courses (90% of courses are in here)
  • The first thing you’ll want to do is verify that the course is in the appropriate table. So in SequelPro, find the course (in whichever table it’s supposed to be in) by Section number (ADC-2020-A). If the course is found, make sure it exists for the correct term (15/CU, for example, is 2015 Summer. FA is Fall and CS is Spring–I’m pretty sure about those but not 100%)
  • If the course is is in regbook_sections_ce then the course should be imported into SVA.edu > course_pages table. All courses in regbook_sections_ce are imported into course_pages, no other requirements need to be met.
  • If the course is in regbook_sections_ft then you have to make sure the schedule_type field=D as only this type of course will be imported.
  • If the course is not in either of these tables, then the problem resides with either the Registrar or Colleague Computing. Either way you’ll want to send an email to Elena Blank and Celeste Barns asking them to investigate.
  • If the course is one of these tables but is not being imported into SVA.edu then try importing courses manually
    • SSH into production
    • $ cd /var/www/html/sva/releases/release/
    • $ ./app/console sva:ingest-regbook-import (no sudo) – this is for the regbook_sections_ce table
    • $ ./app/console/sva:set-undergraduate-credit-courses (no sudo) – this is for the regbook_sections_ft table
  • Either the command will run successfully, or more likely if it isn’t working, it will throw an error. If it gives a permissions error, see the fix to cron events below. Otherwise debug normally.
  • If you need to start debugging anything related to CE, first know whether you’re dealing with a regular ce course or full-time course. Then go into the appropriate ModelBundle/Command (either ingestRegbookImport or setUndergraduateCreditCourses) and from there you’ll most likely find yourself in ModelBundle/Service/SearchCourse…
Symptom: A cron event, such as ce courses are not importing or feeds/tweets are not updating
Prescription: Most of the time this is a permissions issue that running the clear-cache script will fix.
  • If clear_cache doesn’t fix it, run $ sudo chmod -R 777 /dev/shm/sva, then $ sudo ./clear_cache.sh again
  • Occasionally the update-feed-sources command will choke on a poorly formed feed url or feed response. This is pretty easy to diagnose, just run the command from the server ( $ cd /var/www/html/sva/releases/release/app/console sva:update-feed-sources ) and watch the output, then disable that feed from within the CMS

Written by ecorriel

June 8th, 2015 at 9:18 pm

Posted in web

Leave a Reply