Reverse Proxy with Cloudflare Workers: Setup Blog in Subdirectory of Existing Static Website

Blog is an essential part to increase your site’s traffic, authority, and search-engine-related performance. With blog, you can inform people about all the things they need related to your site or business, and then drive them into your website, hence, make them your customers. Regarding search engines optimization (SEO), there are several technical way to serve your blog’s site in relation to you business’ site, which are:

1. Use same domain but different subdomain (e.g. business’ site is example.com, blog’s site is blog.example.com).
2. Use same domain and same subdomain, blog in certain subdirectory (e.g. business’ site is example.com, blog’s site is example.com/blog).
3. Use different domain for business’ site and blog’s site (e.g. business’ site is example.com, blog’s site is other-example.com).

For every option above, there will always be different pros and cons in term of SEO. If you want to know in detail which one of those options that is more effective for your case , you can read this excellent post about it here.

Background Case

To demonstrate how to setup blog in subdirectory using cloudflare workers, I have my existing personal project here as an example. It was a unit conversion static website created using HTML, CSS, and Javascript of which I stored all those static files in Google Cloud Storage as the website’s host. Here are relevant technical stacks for the project:

1. HTML, CSS, Javascript files,
2. Google Cloud Storage to store the files and act as website host,
3. Google Tag Manager to manage head script tags (e.g. Google Analytics script tag)
4. Cloudflare as Content Delivery Network (CDN)
5. Blog powered by Blogger

TL DR; The project is already published for one month now and still lacking traffic, so I decided to try to increase its traffic by creating blog. Since it was a new domain, it has zero domain score metric, which means the website’s authority is very bad. So then I have to choose between these three options that work best for my project. After all the research I did to decide it, I chose the second option which is to serve my blog in subdirectory. It was because:

1. Other subdomain blog will give no profit for me since even if I created it and generate a lot of backlinks from that subdomain to my main site, the subdomain doesn’t have domain authority from the start either, and the backlinks will have very little to no value at all for my main site. This was because different subdomain is considered as different site.
2. Other domain blog will also give the same result as point 1 above if I create new domain just for the blog.
3. The better way for point 2 is to submit my blog to other blog submission sites like Medium here. The only downside is that a lot of blog submission site doesn’t allow “dofollow” anchor, which means when search engines crawl the blog post that contain backlink to your site, it won’t follow the anchor, hence, it won’t crawl your site. But this is not that bad, it will also be counted as backlink, but not as good as “dofollow” backlink.
4. Building great contents, put it in project’s subdirectory is the best way. Because, if people starting to notice the contents, it will directly affect the main site authority because the blog and the main site was under the same subdomain after all.

A lot of digital marketing consultant will also tend to give advice to put blog in site subdirectory for its advantages, but sometimes, technical effort for this purpose is rather complex, especially for static website, so a lot of people choose to make their blog in different subdomain instead.

So, who is this article for?

This article is for anyone who wants to set up their blog under same directory as the main website. I hope the content of this article will work out well for anyone who use the same technical stacks with my project, but if you use different things, I hope this give you some ideas and hopefully you can apply it with your own.

The “How” Part

The main idea is to use Cloudflare Workers as a serverless application platform, to serve a client request for specific site’s subdirectory, and respond to that request with rewritten response from source site. In this case, the “subdirectory” is where you want to put the blog (in my case, I want it to be in https://foreach.id/blog/), and “source site” is where your blog is currently being published in (in my case it was https://blog.foreach.id). You may notice that my “source site” is already in other subdomain of my domain. It is a necessity to firstly publish your blog in custom subdomain of yours because we will need it to have full accessibility of the “source site”.

Step 1

To execute that main idea above, firstly you should create a cloudflare worker. Cloudflare Workers is a serverless platform provided by Cloudflare and it has a lot of great functionalities you can use. Here is the examples of those from the original documentation.

If you never use cloudflare workers before, here is steps to create one.

1. Open your cloudflare site’s dashboard, and choose “Workers” tab.
2. Click “Manage Workers” button.
3. Click “Create a Worker” button.
4. Write, test, and deploy your worker code. You can also edit your deployed worker later. Cloudflare Worker supports several programming languages (e.g. javascript, C , C++, etc).

Open “Workers” tab and then “Manage Workers”

Click “Create a Worker” Button

You can write your code in orange box section, testing the code in blue box section (send request, check the response, check the logs, etc), and when you ready you can click the “Save and Deploy” button to deploy the worker.

For our first worker, we want it to respond for request to our subdirectory with the response from the source site. In my case, I want a request for https://foreach.id/blog/ to be responded with the same response if a client requesting for https://blog.foreach.id. The example provided by the original documentation has one with this exact purpose here.

For my case, the code would be something like this:

addEventListener("fetch", event => {
    return event.respondWith(fetch("https://blog.foreach.id"))
})

This code here means that whenever we make a request to the worker, it will give a response that is similar with a response we get whenever we make a request to “https://blog.foreach.id". To test it, you can use the http request section (blue box section on the image above) to send the request and get the response which will be similar to the response from “https://blog.foreach.id". In the input box on the left of “Send” button, is a url you have to make a request to to run the worker.

Now we want to set it up so if a client make a request from “https://foreach.id/blog/”, it will run the worker we created before. Here is the part when we have to set the route for the worker. To do that, just go to “Workers” tab in cloudflare dashboard, and then click “Add route” button. A dialog box will show up and you can write up any route you want to work for specific worker (choose the worker you want from the dropdown). My worker’s name to apply to the routes is bold-surf-4f3d.

Click “Add route” to add specific routes for your worker to run. After you add routes, it will shows in the list.

A dialog box will show when you click “Add route” button, fill it up based on what you need.

In my case, since I want worker above to run whenever a client accessing “foreach.id/blog” or whatever routes inside “foreach.id/blog” subdirectory, I add two routes for the worker, which are: 1. *.foreach.id/blog , and 2. *.foreach.id/blog/*. The asterisks works as a wildcard to create dynamic pattern to match any URLs that has similar pattern.

To this point here, whenever I access “foreach.id/blog” from the browser, it will respond with similar response from “blog.foreach.id”, which means my blog will appear. The problem is that because the response is still raw and it doesn’t mean to be for “foreach.id/blog”, the functionalities inside that page won’t be completely functional. If you ready to make it all work well, let’s move to step 2.

Step 2

Here in this step, we want to make sure that all the content of the html file we get from the source site is applicable to its new site. I want all the anchor tags in my blog doesn’t direct it to blog.foreach.id/* but instead to foreach.id/blog/*, I want it when I share my blog in social media, it doesn’t share the source site but the new site, etc.

So the main idea here in step 2 is to rewrite some essential tags (anchor, meta, image, etc) from source site’s HTML file to make it works with the new site. For this purpose, Cloudflare Workers original documentation has already provide the example which you can read here. To do this, we can edit the previously deployed worker before and make it to be something like this below.

const OLD_URL = "https://blog.foreach.id"
const NEW_URL = "https://foreach.id/blog"
async function handleRequest(req) {
  const res = await fetch(req)
  return rewriter.transform(res)
}
class AttributeRewriter {
  constructor(attributeName) {
    this.attributeName = attributeName
  }
  element(element) {
    const attribute = element.getAttribute(this.attributeName)
    if (attribute) {
      element.setAttribute(
        this.attributeName,
        attribute.replace(OLD_URL, NEW_URL)
      )
    }
  }
}
const rewriter = new HTMLRewriter()
  .on("a", new AttributeRewriter("href"))
  .on("img", new AttributeRewriter("src"))
  .on("link", new AttributeRewriter("href"))
  .on("meta", new AttributeRewriter("content"))
addEventListener("fetch", event => {
  let url = event.request.url.split("/blog")[1]
  event.respondWith(handleRequest(OLD_URL+url)
  .catch(err => new Response(err.toString(),{status:500})))
})

This code above is very similar to the original documentation of cloudflare workers to rewrite links, I just added some other tags to rewrite and some logic to parse the requester url to make this worker dynamic and applicable with the asterisks in the route.

Here is what happened in the code above. Whenever the worker being requested (i.e. https://foreach.id/blog* accessed by a client), the worker will fetch HTML file from the source site (i.e. https://blog.foreach.id*), and then rewrite some of its tags’ attribute which contains “OLD_URL” and replace it with “NEW_URL”. Tags and their attribute that being affected here are:

1. href attribute from anchor tag (<a />)
2. src attribute from image tag (<img/>)
3. href attribute from link tag (<link/>)
4. content attribute from meta tag (<meta/>)

You can also append or prepend anything to every tags from the HTML source. For example, if you want to add Google Analytics script, you can append it to the head tag by using element.append inside “element” method in AttributeRewriter class above.

This step here is a little tricky so my code here maybe doesn’t completely works for you. You can write up more code to suit you better and you might need more code to modify response data or / and modify request data. There is a lot of factors that can influence the complexity of this step such as the blog template, CORS, etc.

Anyway, at this point, you should be able to:

1. Access blog in your site’s subdirectory.
2. Click any anchor tag and not redirected to the source site.
3. Share your blog and share url of the new site, not url of the source site.

Your blog is already placed in your site’s subdirectory, but there is still one thing to adjust.

Step 3

You want to move your whole blog in you main site’s subdirectory, you don’t want your source site to be accessible because it will hurt your newly-placed-blog popularity. I mentioned before that we will need our source site to be fully accessible, we need it for this step. Here in this step we are going to serve all request for the source site by redirecting it to the new site. To do this you need full access to your source site. And you can do it by simply use page rules or using cloudflare worker again. If you choose to use cloudflare worker, you’ll need to create new worker and then assign routes for it. You can see the original documentation for redirecting request here.

Here is the worker code for redirecting looks like for my project:

const base = "https://foreach.id/blog"
const statusCode = 301
async function handleRequest(request) {
  const url = new URL(request.url)
  const { pathname, search, hash } = url
  const destinationURL = base + pathname + search + hash
  return Response.redirect(destinationURL, statusCode)
}
addEventListener("fetch", async event => {
  event.respondWith(handleRequest(event.request))
})

After you save and deploy the worker, you’ll have to assign routes of which the worker will run, which is the source site. For me, I assign *blog.foreach.id and *blog.foreach.id/* for this worker. Those are the source site of my blog. After save, deploy, and assign routes, you can test if the worker works properly by accessing your blog’s source site and see when it redirects you to your new site.

That’s it !

After all those steps, your blog now will be under your main site’s subdirectory, and hopefully everything work well!

Simple Performance Comparison

At first I think this method will hurt my blog’s performance since it needs to rewrite the HTML file first before the worker deliver it. So I ran a little performance comparison between the source blog site and the new blog site using Lighthouse and the results are unpredictable. Here is the screenshots of both the audits of one article in my blog.

Lighthouse audit for source site (https://blog.foreach.id/*)

Lighthouse audit for new site (https://foreach.id/blog/*)

Seems weird to have the new site scores bigger in performance than its source site. It seems some factors affect it, but my point here is that, Cloudflare Worker is really fast. Because I thought the method will decrease my blog’s performance, but it’s not.

Conclusion

To increase your site’s traffic, authority, or SEO-related performance, sometimes you want to put your blog under your main site’s subdirectory. But there has been a lot of concern about its complexity especially if the static files are hosted in some static cloud storage. You can set it all up by using cloudflare workers to do this three main steps:

1. Respond for request to your subdirectory with the response from the blog’s source site,
2. Rewrite some essential tags (anchor, meta, image, etc) from source site’s HTML file to make it works in subdirectory,
3. Serve all request to the source site by redirecting it to the subdirectory.

This method here has proven to be working for me to move my blog from here (source site) to here (blog in main site’s subdirectory). And there hasn’t been any performance issues resulted from this method either. All is working well.

Note If you find this article helpful, feel free to share it or give it a like. Or if you have some questions, advices, or concerns, please write it on the comment section. Thanks!