Archive for the ‘Web’ Category

Creating a Membership List in Drupal 11 with Aggregating Views

Wednesday, July 9th, 2025

I’ve written before about our use of Drupal for the Southampton Orienteering Club website. We’re now on Drupal 11, and my opinions haven’t really changed. Upgrades are still painful, particularly the community modules that we have to leave behind each time. The user experience for creating content also lags behind newer alternatives. We have a significant amount of historical content on the site (not all of it publicly visible), making a move a daunting proposition. In the meantime, as this post demonstrates, we continue to utilise the powerful features that Drupal and its ecosystem offer.

We had a requirement to provide a membership list for use by the club’s members, which would provide names, approximate home location (to facilitate lift sharing), and a contact mechanism. Previously, it fell to the membership secretary to create this list manually; however, given that nearly all members have an account on the website, it felt like there was a better way.

We already had a permission role that was granted to club members (allowing them access to the members’ area), so it was trivial to create a page that listed all of the website users in that role (and limit access to the list to those in that role). Drupal lets you add custom fields to the user profile. We already have fields for forename and surname, to which I added a location field, which we populated from the old membership list.

User profile

Drupal also has a built-in mechanism for users to contact one another. Users can select the user they wish to contact and provide a message, which is then emailed to the recipient with the originating user as the sender. This has the benefit that users see messages where they are most likely to notice them (in their inbox rather than in some additional system), but without having to expose everyone’s email address to everyone else, which was an area of concern. Better still, users can indicate in their profile whether or not they wish to be contactable.

Contact form

So far, so good. We had a list that showed members’ names, locations, and a link to their contact form if they hadn’t disabled it. The last thing we wanted to add to the list was some additional data for each member, highlighting honorary members, any qualifications (e.g., first aider or coach), and any posts they might hold (e.g., secretary or chair).

We already had a Drupal node type to represent a post, which is then linked to multiple users. This was being used to generate the committee page. I decided to extend this to cover the other scenarios. Drupal views allow you to specify reverse relationships, so for each member, it would retrieve all of the ‘posts’ the member held. Unfortunately, it then renders this as if it were an outer join in SQL, with multiple rows in the table for a member, one for each post.

This is where the Views Aggregator Plus module came to the rescue. Once installed, I could select the “Table with aggregation options” format for my Drupal view. Getting the correct settings was then a bit finicky. I had to add a hidden field with the user’s UUID. I then configured the view to group the post holder relationship using the “Enumerate (sort, no dupl.)” function and group the UUID using “Group and compress” as shown in the following screenshot.

Table with aggregation options settings

The module is significantly more powerful than this. It will, for example, allow you to perform operations such as COUNT, MIN, and MAX on the aggregated rows. That’s maybe for another day!

One further tweak was then needed. The table was styled differently from all of the other tables on the site. Rather than try to replicate that styling, I changed the class in modules/contrib/views_aggregator/templates/views-aggregator-results-table.html.twig from table to views-table.

The final list (or at least the important section of it!) then looks something like the following:

Membership list

WordPress is broken by PHP in Jammy update

Saturday, August 3rd, 2024

This blog has been a bit neglected for the last few years. I miss the opportunity to reflect on something I’ve done and write up those thoughts. We’ll see whether this is a one-off or the start of something new!

The first task was to make sure everything on the site was up-to-date. WordPress does a pretty good job of automatically applying security fixes but the Ubuntu VPS needed an upgrade. The update to Jammy went smoothly enough but attempting to access the site showed the WordPress PHP source. The enabled modules for Apache showed a couple of broken symlinks to PHP 7. After enabling those for PHP 8.1, I saw a WordPress error page: There has been a critical error on this website.

The WordPress PHP compatibility matrix indicates that there are still exceptions with PHP 8 versions. Time to get PHP 7 back…

Then re-enable the PHP 7 Apache modules:

With the site rendering again, I thought I was done but on posting this entry the dreaded critical error reappeared. Looking again at the Apache error logs, /var/log/apache2/error.log revealed errors in lightbox-plus and crayon-syntax-highlighter of the form Compilation failed: invalid range in character class. From PHP 7.3, hyphens need to be escaped in regular expressions. I could have rolled the PHP version back further but decided to patch the offending files. (I probably need to review the plugins in use on the site and remove those that are no longer supported.)

And, finally, we’re back in business!

Update time

Thursday, September 5th, 2019

WordPress has been nagging me for some time that I needed to update the version of PHP this blog is running on and, in particular, Jetpack had finally given up on me. The sticking point has been that the Digital Ocean droplet it’s running on has been stuck back on Trusty Tahr and a previous attempt to upgrade it had gone awry.

I finally took the plunge and set up a new droplet running Bionic Beaver and eventually found the right combination of PHP modules to get everything running again. Whilst I was at it, I ticked another item off my todo list and enabled TLS (trivial with the aid of Certbot and Let’s Encrypt). A late evening but nothing too painful.

On the downside, when I first set the blog up (back in 2005) I used Gallery to manage images. The WordPress plugin died a while back but the Gallery install itself failed to play nicely with the new PHP version. As a consequence, the item to write a script to locate all those <wpg2id> tags and replace them with the appropriate images still remains very much on my todo list. Oh, and then there’s all those GPX files that were being displayed with Google Maps…

Website backup to pCloud

Wednesday, January 30th, 2019

Another SOC website related posting – this time on the subject of backup. The website is backed up by the club’s current hosting provider (Krystal – who, a year in, I can highly recommend) but I was informed that the club had bought a large quantity of cloud storage for the purpose of storing its map archive and, for belt and braces, it made sense to also include backups of the website there.

As it turned out, the cloud storage was courtesy of pCloud who are best described as a Dropbox clone i.e. the expected interaction patterns are via the web UI, mobile, or sync from the desktop app. A quick search turned up rclone which describes itself as “rsync for cloud storage” and, amongst the list of supported backends, includes pCloud.

Install on hosting provider was straightforward. The configuration process is interactive (opening a browser to log in to pCloud) but the docs also cover how to create the configuration on one machine and copy them across to another. A copy is then as simple as:

I started out looking to use drush arb to create a backup but, as the same hosting is used for a WordPress site, it was easiest in the end just to write a script using tar and mysqldump to create the archive of the file system and database tables. This is then triggered nightly on a cron job. Each backup is around 0.5GB so I wasn’t too concerned about incremental backup and, with 2 TB of storage to play with, it will be a while before the question of cleaning up old backups comes back to haunt me!

Drupal 8 Migration

Monday, January 28th, 2019

For my sins, I have now been involved in the management of our orienteering club’s current website for over 10 years now. Back then, we wanted to make it as easy as possible for club officials and members to contribute content and, after evaluating WordPress, Joomla! and Drupal, we went with Drupal as our Content Management System. The extensibility of Drupal makes it immensely powerful but, as with many open source projects, the rich ecosystem of contributed modules can be both a blessing and a curse.

Although the details have been long forgotten, I do remember that the move from Drupal 6 to 7 was a painful one and so, despite it being over three years since Drupal 8 was released, I was in no rush to migrate. In the end, it was a security vulnerability in one of the modules that wasn’t going to be addressed in v7 that precipitated the move.

The major changes in core Drupal have seemingly been too much for many module contributors to make the move. An initial assessment wasn’t particularly promising: of fifty-five non-core modules the current site had installed, five were no-longer needed in Drupal 8, six had GA v8 versions and a further fourteen had beta versions available. A migration estimate site put the effort involved at several weeks worth and, in the end, it probably wasn’t far off!

My first task was to slim down the number of modules installed. Many weren’t actively in use any more (e.g. content_access and views_data_export) and others had simple replacements which had easier migration paths (e.g. swapping out timefield for a simple text field). Ironically, the module with the security flaw was one of those that I disabled but, having started down this path, I was determined to complete a migration.

It was then time to start the actual migration. Thankfully the process now involves setting up a parallel site as it would still be weeks before I had anything that was approaching usable. One of the issues was that no private file path was set up during the migration. Another, that the migrated text formats were using a handler that no longer existed. Opening and resaving them fixed that problem. Another of the random error messages required manually modifying the database to remove the upload field from entity.definitions.bundle_field_map in the drup_key_value table (go figure).

The site makes extensive use of custom content types and views which are finally a part of core Drupal. Views are not part of the default migration though, and, in the end, I just recreated them manually. The same was true of all the patterns for pathauto.

At this point, with the styling also re-introduced, the site was ready to go live again but there were still problems waiting to be found. One was that, what used to appear as a date field, now appeared as a datetime field in forms. In the end, I decided to test out the new REST capabilities to export the contents of the field and reimport into a new field with the correct type. The only catch here was that there is no querying capability in the REST API so it was necessary to create a JSON-rendered view that listed the required nodes in order to retrieve their ids so that they could then be processed one-by-one. The rest was just a short bash script using curl and jq.

Hopefully, the migration can now be considered complete. The site now uses relatively few custom modules which is, undoubtedly, a good thing for future stability. If the move to Drupal 9 looks anywhere near as painful though, I now know how to extract the entire site content so maybe it will be time to revisit the CMS landscape. It would hate to think that I’ll still be debugging PHP errors in another ten years time!

ES2015 in Production

Thursday, April 21st, 2016

Bård Hovde gave tonight’s Developer South Coast presentation on the subject of “ES2015 in Production” (or “ES6 in Production” if you must). You can find the slides here with the source for the presentation over on Bård’s GitHub account. He did a great job of making the subject matter entertaining. Beyond being able to say goodbye to all of that boilerplate, my main takeaway was the use of Babel for transpiling ES2015 into ES5, so no excuses about waiting for browser compatibility! The Babel site also has a nice overview of ECMAScript 2015 features.

Back online

Sunday, March 20th, 2011

Shortly after moving my blog to an EC2 instance I received an ominous email stating:

We have noticed that one or more of your instances are running on a host degraded due to hardware failure.

The risk of your instances failing is increased at this point. We cannot determine the health of any applications running on the instances. We recommend that you launch replacement instances and start migrating to them.

After a bit of searching around it seemed that, as my instance is EBS backed, I should just need to stop it and restart it and, in all likelihood, it would move to another host. Sounded simple enough but the stop seemed to be taking forever. After downloading the command line tools I tried to force the stop but still no luck – except when I logged in again the following night it was finally showing as stopped. Unfortunately, when I tried to start it I received the following error: “Server.InternalError: Internal error on launch”.

I posted a question on the EC2 forum and, as you can see, was told that the root volume was in an “abnormal attachment state”. You’ll also see that my attempts to force a detach via the AWS Management Console appeared to fail or at least, the console thought the volume was still attached and therefore wouldn’t let me reattach it. Reverting to the command line utilities again allowed me to successfully detach and reattach the volume and then start the instance. Another delay whilst the DNS refresh took effect (perhaps I should try out an Elastic IP) and, about three days after I first hit stop, the site is finally back online. I’m putting it down as a learning experience!

Server hacked

Wednesday, March 9th, 2011

When attempting to post the image for my last blog entry, it failed to be resized. When I logged in to the server to see what was up with ImageMagick, I didn’t appear to have permission to execute ls. Or ps. Or netstat… A quick Google suggested that these were the hallmarks of a rootkit attack. Unfortunately the files had been modified prior to the oldest Slicehost backup that I had. At this point I realised the server was still running Intrepid, limiting my chances of picking up packages to detect and remove rootkits. After a reboot of the server I discovered that I had lost all connectivity. Booting up a Slicehost rescue image I was able to retrieve all of the data I needed. Now to get things up and running again. Earlier in the year I had been playing around with a free micro-instance on EC2 and this seemed like the ideal opportunity to switch across. The instance is running Apache rather than nginx as on my Slicehost image. This needed a bit of tuning down to prevent segmentation faults. Everything seems to be running smoothly now. I just need to switch the DNS records away from Slicehost and then I’m done.