Updating a Legacy PHP Web App Part 3: Create a Dev Environment

In part 1 of this series, I discussed the decisions behind not throwing the code away entirely and getting the app running. In part 2, I broke down the steps to understanding your application logic so you can start documenting follow-ups. . In this part, I will continue to explain the technical refinements for a stable working environment that you can consistently dig into everyday.

Docker-IZE

You now have the app up-and-running locally! …But if you change your hosts entries or have another project that requires Apache with mod_rewrite specifically OFF, that’s not going to be fun to figure out. Containerization is a great option and Docker is the primary technology. If you took decent notes when you set this thing up, you should be able to easily figure out your little Docker cookbook. I’m all about docker-compose. My web container and MySQL container need to talk to each other and this is easily configurable in docker-compose. I can attest that containers works great for both new and legacy systems. It’s a great way to isolate your coding work from your personal work and it feels great to know that it’s almost impossible to break your dev environment by making unrelated changes.

From just a few Google searches, I was able to find Dockerfile and docker-compose.yml combinations that gave me everything I needed with the added benefit of a phpmyadmin installation. See the simplicity of the Dockerfile below:

Dockerfile

FROM php:5-apache

RUN docker-php-ext-install mysqli

RUN a2enmod rewrite

Now I am able to start up and shut down my dev environment with the ease of executing docker-compose up (or down). If you’ve ever taken a break from a project and tried to return to it again, then you know about the pain of getting your local dev environment up and running after a few months of using your personal computer. With the Docker-based solution, you’ll never run into this problem again. Believe me, I disliked Docker at first. Before the days of docker-compose, it was really hard to figure out how multiple containers can use the network to talk to each other. It’s also really cool to have developer set up directions that are a single line!

Set up a remote test environment

For you, For your team, for your client

Your client might be ok with seeing changes directly in Production but for the sake of your own sanity and not destroying everything all at once, why don’t you set up a remote test environment so you can deploy your local work there and do some QA testing. Besides, if you are going to end up doing the massive amount of refactoring you think this charming, older web app will need, you will want a remote test environment up and running for your client, your eventual outsourced remote dev in Bulgaria, or anyone else to tinker with as they QA your work. I, personally, like to QA my work on my local machine and also on the test environment. I regularly refresh the test environment with production data and production content as it’s much easier to copy it all over because they are hosted on the same machine from different docroots. Besides, at this point you’ve already set up a database on your local dev environment so you know the ins-and-outs of database exports and imports and you know which tables give you trouble. The test and production databases are completely separate and have no visibility to each other.

Environment	Local	Test	Production
Purpose	Writing new code Debugging Refactoring research & discovery analysis local QA	Reproducing steps for bug reports integration environment for code from other developers viewing Production data in the app outside of the Production environment testing in the shared hosting environment	For customer and admin users only

The remote test environment quickly became a great place to integrate the work that I did with the work I selectively assigned to my Bulgarian teammate. I hired him from Upwork and gave him a few technical tasks to assess his skills. When I opened bug reports for him, I gave a URL and reproducible test case from our test environment. When we had Production issues, I would reproduce them in the test environment to be sure. This would help ensure that both of us would stay away from the Production docroot on the web server.

Conclusion

If you’ve made it this far, you have a well-running local dev environment and a remote test environment. You have GitHub set up as your source control with a technical wiki and a collection of issues documenting your next steps. This concludes the second part of this guide. In following installments, I’ll document setting up the PHP Remote Debugger to enhance your debugging experience and I’ll further discuss some methodology to slowly but surely modernize the application architecture. I’m looking forward to your feedback on this article! Do let me know about your trials and tribulations with older, less structured web apps!

Updating a Legacy PHP Web App Part 2: Understanding Your App

In part 1 of this series, I discussed the decisions behind not throwing the code away entirely. In this part, I will jump right into the technical steps to establish a stable working environment that you can consistently dig into everyday:

1. Just Get the Damn Thing Running

2. Start Taking Notes

3. Understand the Structure

Step 1: Just Get the Damn Thing Running

The sooner you can easily relate the screens you are seeing on your web browser to the files that correspond to these screens, the sooner you’ll be able to fix broken stuff! Don’t overthink your development environment’s requirements and your operating system and things like Docker. Just try to get to the login page on your localhost.

A good hint that you need to active Apache mod_rewrite is to look at the .htaccess file in the docroot.

This particular web app was running on a shared hosting service on Apache talking to a MySQL database. Hey, I’ve set up Oracle installations on Windows 2000 back since Oracle 8i was distributed on CD-ROMs. I can handle a MySQL install with a basic Apache server. Au contraire, mon ami. It’s not this application was terribly complicated, it’s that without proper documentation, the only way to figure out how get this Login page to show up is going to be to take it step by step. At first, I saw that I could not hit any URLs at all and I was getting all sorts of errors about redirecting and I figured out I need the old school Apache mod_rewrite module enabled. So now I can hit URLs and I was hoping to get a simple login page or some error telling me it cannot find a MySQL connection but I was just getting a white screen of death.

I had to look at the code to see that this web site was looking for two very specific domain names: the custom localhost the previous developer put in his hosts file and the production domain. Luckily, I could configure both on my machine so that I could get some static content to load.

Localhost logic found in: docroot/inc/template.php

if(strpos(getcwd(),"/Users/Adam/")!==false) define('LOCALHOST',true);

Well, this isn’t going to work. My name is not Adam and, well, I’m not developing on Windows. Also, an example of misplaced logic: system bootstrapping in the template engine.

My next step would be to export the Production database to my local computer. While there were some weird issues with all of the HTML content stored in the cms table, and ensuring I wasn’t wasting time exported tens of thousands of rows of user data, I was able to create a good enough copy to get started. This is all except for the fact that I could not access the database without executing this command:

SET GLOBAL sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''))

Forgive me for not remembering the exact error that occurred as I have not experienced it in more than one year. I wrote in the technical wiki that this was a required step in the database configuration. This brings me to my next step:

Step 2: Start Taking Notes

README

It can be very frustrating trying to get the damn thing running on your computer. Taking notes is usually the last thing we care about. As you inch towards progress, we often forget the steps we took to get there! Do yourself a favor and create a simple text file where you can jot down quick and reasonably legible notes. Eventually some of these notes might become part of your formal technical specs. (Yes, you should write those too).

deployment directions

Other documentation you should consider writing is how you would like to formalize deploying code for Test releases and Production releases. It might be as simple as pulling from Git in a temp directory on your remote server.

If you are planning on expanding the team or even taking a break from the project for a while, it’s these sorts of fundamental steps that differ between projects that are hard to pick up when you need to do something quick like a small bug fix.

Create a guide for a new developer

You do not need to memorize all of the details about the oddities of this web app. By writing it down in a wiki, you’ve given yourself a searchable tool and the gift of sharing. Even mundane things like the ssh command to the Production server or the URLs of the various environments will prove useful in the future. The less about this app you have to memorize, the better for everyone. See the screenshots and associated captions of documentation examples below:

View fullsize

Wiki Home Page

View fullsize

Environment List

View fullsize

Wiki contents

View fullsize

Tech Specs

Step 3: Understand the structure

FIle Structure, config, get this into source control

You’ve already dabbled around the app configuration to connect your database. I’m sure there are many other configuration variables. Is the config centralized? In my case, there was config for:

• cookie domain.

• MySQL connection.

• localhost detection.

• CMS images location (while CMS HTML is stored in the database and reference relative image URLs, the path of the images ended up being plugins/Xinha-0.96.1/plugins/ExtendedFileManager/demo_images. If you see humor in this as well, we can be friends).

• TEST_MODE detection (no, this is not for testing the app. We discovered this was for Adam’s experimental features).

At this point, you want to document the directories were user content (pics, thumbnails) live so you do not end up checking them into source code. You don’t want to check in 1.6 GB of data into GitHub. That’s not fun. It’s time to move to GitHub.

Check it into GitHub and bam!

If you are still updating a text file, it’s time to move your documentation into the wiki, set standards for source control, branching and deployments. You might not be on this project as long as you think and it’s good to leave projects in better condition than you received them.

Example of directory structure

|- css
  | events.css
  | profiles.css
|- inc
  | events.php
  | profiles.php
  | template.php
|- templates
  | events.html
  | profiles.html
.htaccess
events.php
profile.php

Application Architecture

Once I found out how the configuration was coded, I naturally wanted to get deeper into the application code. I discovered that what seemed like a complete mess of 200 PHP files did actually have some form of MVC architecture behind it. In the root directory of the project folder, files with names like “events.php” and “profile.php” existed. Inside a folder called “templates/”, I found subsequent “events.html” and “profile.html”. In the “inc/“ folder, I found another set of “events.php” and “profile.php” files. Similarly in the “css/“ folder, I found files with the same names. With the understanding that all hits to the web site were touching files in the root directory of the project and that most of the other project folders were restricted from serving content according to the .htaccess file in the project root, I was able to determine that the root directory files served as endpoints and controllers and view generators.

In Java, we would have called these classes things like EventsController or put them in a package of controllers, but don’t get me started with my Java nostalgia.

Hitting local.mywebsite.com:8001/events touched /events.php whose source code included PATH ./inc/events.php and /css/events.css/. In the same file, the Template engine performed a merge on the output from the database with the HTML template file using PHP’s str_replace(). I did find a semblance of the View of the MVC paradigm. /inc/events.php ended up being a combination of the Data Access Layer and the Service Layer for all things “events”-related. The original developer did have some forethought to separate the SQL from the HTML rendering, although this ended up being quite inconsistent.

Below is a slimmed-down example of how a list of events is populated. After spending quite a bit of time understanding how this app was built, I gained an appreciation for the elegance of it given the lack of official MVC framework.

<?php
define('PATH', '');

require_once PATH . "inc/template.php";
require_once PATH . "inc/util.php";
require_once PATH . "inc/events.php";
    
// Retrieve the template
$html_body = file_get_contents('templates/events.html');

// Pull out the template mask
$tpl_event_item = Util::template_segment($html_body, '<!-- EVENT_ITEM_START -->', '<!-- EVENT_ITEM_END -->');
    
// Data Access Layer
$result = Events::getUpcoming();
    
$event_list = "";
while($row = mysqli_fetch_assoc($result))
{
   $tmp = $tpl_event_item;
   // Fill in 
   $tmp = str_replace('{{EVENT_TITLE}}', $row['title'], $tmp);
   $event_list .= $tmp;
}
    
// Fill in the template
 $html_body = str_replace($tpl_event_item, $event_list, $html_body);
    
// Override styles
Template::style("
   .new__style
    {
      border-radius: 3px;
    }
");
    
// Include stylesheet
Template::css("/css/events.css");
Template::js("/include/a/js/file/here.js");
Template::script("console.log('output any JS you want here');");
    
// Send the HTML back to the browser, we're done here
Template::mkT();
    
?>

While there were plenty of other discoveries I made that baffled me (like how some of the template HTML files for core features were stored in the database), I was able to find enough good patterns in the code so that I could find a way forward and, more importantly, create follow-ups for code improvements (put it in the wiki!). Let’s start with “all templates should be HTML files checked into GitHub”.

Conclusion

By this stage of the code discovery process, I started feeling more confident that any mysteries I would encounter would be solvable. The next phase of the project setup is to create a stable and reusable local development environment. By choosing Docker technology, I set my self up for a possible future of using Docker in Production as well. See part 3 for how I tackled these next steps.

Updating a Legacy PHP Web App Part 1: Introduction

Taking the cover off of a legacy PHP web app need not be so painful. I spent the last year traveling around the world, which you can read about on my travel blog. I started my own software engineering consulting firm, digital nomad style and I took work with me. I figured it would be a good idea to keep some flow of income while I was enjoying hiking and beaches and lakes and mountains. I learned a lot of lessons about being a contractor while I was far, far away from anything reminding me of home. Namely, I was pushing out some complicated refactoring work from Malaysia, Fiji and Samoa. Communicating with my clients about their changing requirements became more challenging as the months went on.  fI felt more distant from the project with the passing of time and increasing geographical distance. As the technical issues got deeper, I realized I was very glad I had made some fundamental technical decisions at the beginning of my work.

This is when the lessons of my corporate years really paid off. I made sure to set up a well-documented development environment that I could easily spin up and spin down so that I would never run into having to reconfigure my dev env should anything happen to my personal laptop. This multipart series will walk you through my experience inheriting this project. Trigger warning: unflattering screenshots.

View fullsize

Welcome to your legacy project! Get used to using the shared hosting service console because it’s all your client is gonna end up handing over to ya.

Mom always said if you don’t have anything nice to say [about UI design], don’t say anything at all.

My New York-based client handed over access to a shared hosting service with a docroot consisting of:

• 200+ PHP files, 20 CSS files, 15 JS files, all split among many directories.

• a few content-only directories with 1.6GB of content-related data.

• access to a Production database.

The PHP code was raw and mostly written in 2009. There was a homemade template framework to spit out HTML files. A teeny, tiny bit of jQuery had been used on the front end. There was no framework, such as Laravel or Eloquent ORM, no  distinct service layer or REST APIs, and no test environment. The previous engineer was testing locally and pushing code to Production. It’s never been my style to wing it like that. To get started, I had no choice but to dig in without developer set up instructions, deployment instructions, or, you guessed it, a source control repository.

“The good part of the story: I got to set all of this up from scratch! The bad part of the story: when you are in need of remote project work, you don’t have the luxury of choosing glamour projects and this project was not glamorous.”

Let this document be a guide for all of the developers out there who inherit a legacy project and feel overwhelmed by the task-at-hand. Of course I know you, dear reader, are not overwhelmed by this feat, you are merely reading for enjoyment. (I can provide validation for you whenever necessary.) As I walk through the phases of my approach, I will also share my technical philosophy where applicable so you can understand why I took these steps.

Goal

This web app is part social networking app with user profiles, along with messaging, group events and user search. Take this ugly old PHP web app and put a brand new skin on it, provided by a designer. In this phase, we are not adding any new functionality but we also cannot remove any existing functionality, even for the purposes of simplification.

Methodology

How do I start?

A snapshot of the table of contents of the Business Requirements document shows that there were more administrative features hidden to the customer than there were features for the customer.

For me, divide and conquer has been a computer science strategy that I use in all forms of my life. When it comes to complex technical projects, it has always done me well. I’m comfortable with breaking up large tasks into smaller tasks.

Back in my days working at HBO, when we wanted to modernize older systems, we would go through a “Buy vs. Build” project phase. This was a drawn out process that came with many benefits. It forced us to really think about what it would mean to throw out the entire legacy codebase and rebuild it from scratch using a vendor application. In the case of this project, I had spent quite a few hours documenting all of the existing functionality in a business requirements document.

Beyond the many idiosyncrasies of the app, like configurable widgets on the user dashboard, a quick view of events (in addition to the standard view) and all sorts of combinations of blocking users, profile visibility, and hiding photo galleries, there was also a slew of administrative functionality that allowed the owner of the company to create customized pages in a CMS (with the HTML stored in the database), alter site data, see how users use the site and distribute emails to large mailing lists of users. Each page also has hidden UI components that displayed only when the Admin is logged in that allows him to see extra information and even update user data. My client set a strict goal that we could not remove any existing features!

There were so many “hidden” UI elements only visible to the administrator, I thought the best way to layout the document would be to include the administrative UI elements in each related feature description and highlight it to give a sense that this is a special use case.

I knew after all of the hours of digging into these elaborate features that there was no way we could rewrite this thing from scratch and keep track of all of the functionality that would need to be rebuilt. While it would be more painful, we would have to dissect this baby piece by piece. I also did an evaluation on the many out of the box social networking packages. Unfortunately, none of them offer the level of admin tools that he currently has. Re-skinning these packages to meet the designer’s layouts would also potentially break the provided templates. I was leaning towards upgrading the app module by module.

Can we throw the whole thing away?

There is a type of developer out there that exists who only enjoys writing new code; their own code. I was never one of these types of coders. I enjoy learning about how another engineer built a piece of software. I often think that some engineers rush to decisions to throw away old code, because they personally enjoy writing new code, without considering the cost associated with it. Without a defined set of features that need to be rewritten, how can you guarantee a complete rewrite to your client? In our case, the list of requirements was so extensive, my estimate for a complete rewrite was higher than divide-and-conquer refactoring. Also, throwing away code that works is generally not a good idea. Working code is the result of person-hours of labor including countless hours of thought.

In my career, I have never thrown away an entire project’s worth of code for any sort of upgrades including an internal HBO app originally written for Mac Hypercard, rewritten in Java, where we extracted the SQL code. We salvaged something and let me tell ya, these were very complicated SQL queries that some group of people had, years earlier, had spent significant time putting together!

Now you understand why I chose to work with this older codebase. It had not received any lovin’ in a long time. In the next segment, I’ll document the technical steps I took to get started doing technical work on this project. I’d be interested to hear your stories regarding making hard decisions about what to do with older code.