Intelligent page preloading with Futurelink

I wrote and released a library that has improved page load times on navigation (not initial page load) by an average of 870ms on our website. It’s on GitHub, free for you to use: SamKnows/futurelink.

We recently released a brand new version of our website powered by Vue.js: you can find it at samknows.com. It’s a single page web application, and pages load super quickly as you navigate between them (initial page load is pretty quick too due to server-side rendering, but that’s not what this article is about). When you click on a link to take you to another page on the site, most of the content already displayed as the HTML, CSS, fonts, and JavaScript had all been downloaded already.

Unfortunately, we were being let down by our images. While all the other content was displaying instantly, the images were taking just as long to download as they would on a classic website where the entire page is downloading every time. If anything, the other content displaying quickly was just highlighting the problem — as the HTML didn’t need to be downloaded, everything but the images was displayed without a loading period, and the browser preparser didn’t even have a chance to give the images a head start.

Here’s a video of what site navigation looked like over a poor connection (4Mb/s down, to be specific — about the speed of a 4G connection):

We wanted the images to load quicker — or at least appear to load quicker. While there’s other things we can do (and will do, eventually) such as using progressive jpegs and optimising the image files with something like imagemin, we decided to start by preloading the images: starting the images used on the next page downloading before the user requested the page.

Preloading images

I investigated a couple ways to preload the images, such as using file-loader to inline images into our script file, or downloading the content of the next page when the user hovers over a link, but they had trade-offs that we didn’t find acceptable.

I decided to experiment with another approach that I hadn’t seen before: tracking the position of the cursor, detecting when it is slowing down (and thus possibly approaching a link), and looking to see if there is a link in the place it is heading towards.

Time for some research! I set up a simple page with five links that had to be clicked on in order, and then tracked the cursor and sent the data back to me for analysis. I sent the page out to some colleagues and friends so that I could see how they used the mouse to interact with the links.

Here is one of the results (this person clearly can’t count):

There is a dot in every position the mousemove event is fired. When the dot is red, that means the cursor is accelerating, and when the cursor is blue, that means the cursor is decelerating. The acceleration was calculated by approximating the cursor speed at the beginning and end of the last few points and using v = u + at, one of the equations of motion.

We can easily see where the cursor is heading and whether it is likely to stop. Let’s only show the cursor events where it’s clear that the cursor is slowing down:

Then, let’s attempt to calculate where specifically the cursor is heading. I did this by calculating the direction the cursor was heading in and the speed it was moving at, and then drawing a point slightly ahead of the mouse.

On the following image, the red points are the actual mouse position, and the green points are where the mouse was heading towards at the respective red points.

By drawing an imaginary line between the red point and the green point and detecting whether it is going over a link, we can detect that the mouse is slowing down to click on a link.

Here’s a video of the detection in real time — this time, by someone who can count:

You can see in this video why I decided not to go for the hover approach mentioned earlier: this approach predicts a future click a lot earlier, and also doesn’t return a false positive if the user is just passing the cursor over a link to get to another link on the page.

It seems to work great! Checking against some of the other data collected confirmed this:

After some frankly horrible mathematics testing whether the line between the red and green points passed over a link — calculated by seeing whether that line intersects with any of the four lines of the bounding rectangles of every link on the page — we had produced a script that watched the mouse and fired a callback whenever it predicted the user was about to click on a link.

(We’ve named the library “futurelink” and open sourced it. More details at the end of the article.)

I then hooked the library into our Vue app. We use vue-router, so this is what was required to do that:

  • Call router.resolve(path) (docs here) with the path returned by the library.
  • Get the matched component from .resolved.matched[n].components.default where n is the last item in .resolved.matched.
  • Save the component in the data object.
  • Render it in a hidden div below the footer of the site using <component :is="matchedComponent"></component>.

It’s quite complicated, and in an ideal setup, there’s a few more steps than that (such as making sure each page is only preloaded once), so I’ve wrapped it up in a Vue component that you can find here: SamKnows/vue-futurelink. You can use futurelink in your Vue app with only a few lines of code!

The results

The code has been live on our website for a couple weeks, collecting data about how accurate it is but not actually preloading the next page — I was worried about false positives and didn’t want to waste the bandwidth of our users. Here’s what we found:

  • After excluding anomalies, futurelink predicted that the user was going to click the link an average of 870ms before they clicked the link, giving us nearly a second to download assets required to display that page. That’s huge!
  • There were 1.43 pages preloaded for every 1 page loaded. This means that after the initial page load, 30% of the bandwidth is used for images that aren’t actually going to be displayed. As most of the size of the page is from the initial load, we think this is worth it.

Here’s a video showing the difference it makes:

Futurelink

We’ve open sourced the library, and you can find it on GitHub at SamKnows/futurelink. There’s also a simple demo here and you can see us using futurelink on our website if you open the network tab in the developer tools. It’s released under the MIT license, feel free to use it!