Magento File Structure and URLs

under Magento SEO.

In order to make changes to your Magento theme and complete other development tasks necessary to achieve your SEO goals, you will need to know your way around the Magento folder structure. To optimize your site for search engines, you will also need to properly optimize the URLs on the frontend of your site.

In this post, we'll discuss file structure and URLs on the backend and frontend of Magento, and how to work with them with SEO in mind.

Backend Folder Structure

magento folders

In order to make any changes to your site's theme or add new modules, you'll need to have a basic idea of how to navigate your Magento installation's folder structure. If you have no development or sysadmin experience, it would be a good idea to leave this section to those with more experience. This is true of a lot of tasks in Magento, unfortunately, so if you are going to be working with Magento, you are going to be getting some development experience.

Bear in mind that this post assumes you are working with Magento 2.

Magento's folder structure includes these folders:

  • app
  • bin
  • dev
  • generated
  • lib
  • phpserver
  • pub
  • setup
  • update
  • var
  • vendor

Let's go over these one by one.

app

A lot of important things you will work with go in this folder. For starters, custom modules and other third party modules are, by best practice, placed in:

app/code/[company]/[module]

The code directory doesn't exist after you've done a clean install, because all of Magento's core modules are located in the vendor directory instead, keeping them separate. (This is in contrast with Magento 1 where they were not kept separated.)

Design related files go in the design subfolder here. Your custom theme for users should be found at:

app/design/frontend

While admin themes should be located in:

app/design/adminhtml

If you are operating a multilingual site, language packages go in:

app/i18n

bin

It's unlikely you will need to do much here often. The Magento CLI script found here is for clearing the cache, reindexing, and so on.

dev

The dev folder stores tools for developers like migration tools and tests.

generated

If your code calls a class that doesn't exist, Magento will generate the class here.

lib

Libraries and system code go here. It's a very good idea to stay out, unless it's to add a new library folder without otherwise touching anything.

phpserver

Just don't touch this folder. The likelihood of you creating massive security holes by accident is very high.

pub

The index.php file that runs Magento goes here. Static files generated by your Magento theme also go here.

setup

The Magento installation files.

update

Contains the updater application responsible for keeping Magento updated.

var

Caching and certain automatically generated class types go here. All files here are temporary and in some cases you will need to clear it for changes to take effect.

vendor

Magento core modules and modules installed through composer go here. Editing this code is heavily discouraged, since changes are intended to be made only through composer. Custom code should go in app/code as discussed above.

Magento URLs

Now that you have familiarized yourself with the folder structure on the backend, let's talk about frontend URLs. URLs play a part in how search engines understand what your site and its pages are about. The way they display can contain useful semantic context and information to help search engines understand how your site is organized and what content is useful when.

Enabling Search Engine Friendly URLs in Magento 2

This setting doesn't do as much for your SEO as some might claim, but it's helpful nonetheless. Basically, it removes index.php from the URL, which only serves as excess text that neither search engines nor human users benefit from seeing, and which may make it slightly more difficult for them to understand.

To do this, start by going to the Stores link in the left sidebar, then click on Configuration in the Settings section of the pop out menu:

magento store configuration

Now, in the left pane, select Web under General. Scroll down to the Search Engine Optimization section and make sure that Use system value is unchecked and that Use Web Server Rewrites is set to Yes. Finally, don't forget to click the Save Config button when you are done:

set magento seo friendly urls

This will remove index.php from your URLs, so that instead of seeing something like:

https://www.example.com/magento/index.php/storeview/url-identifier

They will see something like:

https://www.example.com/magento/storeview/url-identifier

Setting The Base URL

The base URL is the URL that appears at the beginning of every URL, that subsequent folders are attached to.

For example, the base URL often looks something like this:

https://www.example.com/magento/

To send the cleanest message to search engines, the base URL should contain as little extraneous information as possible. We recommend including a "magento" folder since this doesn't tell search engines anything about the page. A basic base URL like this is typically what you should go with:

https://www.example.com/

There may be some circumstances where you will want to place all of your Magento frontend in a subfolder, for example if you also operate a blog and want to use a subfolder to indicate whether users are at the store or the blog. If so, we recommend using a subfolder that communicates this to users and search engines clearly, such as:

https://www.example.com/store/

To set the base URL in Magento 2, navigate to Stores > Configuration > General > Web as discussed in the SEO friendly URLs section above.

Now scroll down to the Base URLs section and click the expansion arrow, and simply edit the Base URL field to include the base URL as you want it to be used on your site.

Make sure to include a trailing slash in the base URL. That is, do NOT use something like:

https://www.example.com

But instead use something like:

https://www.example.com/

magento 2 base url

Below the Base URLs section is the Base URLs (Secure) section. Make sure you edit the secure base URL to match the "unsecure" base URL. The secure base URL is used whenever the user is entering sensitive information. We highly recommend using https (not http) for both the secure and the "unsecure" base URL, as site-wide https provides a slight ranking boost in Google search results, and sites that don't use https now display perpetual warning signs in Google's Chrome browser.

magento 2 secure base url

Removing The Category From The URL

Ideally, the category folder or folders for your products should help search engines better contextualize the products and turn them up for searches that include "head" keywords or where Google understands that the searcher is looking for something that would belong in that type of category.

Including the category URL is theoretically useful, then.

However, category URLs come with a set of problems as well, and unfortunately the problem is big enough that it is often worth removing the category from the URL altogether.

The problem is that, in Magento, by default, the product is accessible from URLs both with and without the category folder:

  • https://www.example.com/category/product
  • https://www.example.com/product

Since both URLs lead to the same page, this leads to duplicate content issues which can dilute your SEO authority as well as create indexing issues where a less authoritative or less relevant URL is the one that gets indexed.

While you could use rewrite rules to ensure that only the version of the URL with the category folder is accessible, there is a compounding problem. If your products are listed in multiple categories, this creates still more duplicate URLs, and there is no clear way to do a rewrite rule, since this results in three or more duplicate URLs with no clarity on which category URL would be most appropriate:

  • https://www.example.com/product
  • https://www.example.com/category1/product
  • https://www.example.com/category2/product
  • https://www.example.com/category3/product
  • ...

The easiest, most surefire way to deal with this problem, then, is to simply make sure that the category folder is never referenced in the first place.

To do this, start by going to Stores > Configuration in the left sidebar menu. Next navigate to Catalog > Catalog in the panel menu on the left hand side of the store configuration page:

magento 2 catalog catalog

Now scroll down to the Search Engine Optimization section and make sure that Use Categories Path for Product URLs is set to No. If it isn't, you may need to uncheck the Use system value box, then click the drop down arrow and change the value from Yes to No. 

magento 2 remove category from url

Don't forget to click the Save Config button when you are done.

URL Suffixes

By default, Magento adds a suffix to product and category pages, like so:

https://www.example.com/category/product.html

While the suffix has minimal impact on SEO, it is extraneous information that doesn't help users or search engines. You can remove it by navigating to Stores > Configuration > Catalog > Catalog > Search Engine Optimization as discussed in the section above on remove categories from the URL.

Now just uncheck Use system value for the Product URL Suffix and Category URL Suffix fields and delete .html from these fields, remembering to click the Save Config button when you are done:

magento url suffixes

Get The Product URL In The Code

In order to reference URLs from within your theme you will need to be able to get the URL using php. There is more than one way to get the product URL using function calls in Magento, each corresponding to a different part of the URL.

Getting The Product URL Key

The URL key is just the last part of the URL, leaving out all folder structure. This is accomplished using getUrlPath(). For example:

$product->getUrlPath();

Would store a string like 'the-product-name' if the complete URL were something like https://www.example.com/category/the-product-name.

Getting The Product URL With The Category Path

To get the category folder as well as the product URL key, you also use getUrlPath(), but you include the category variable. For example:

$product->getUrlPath($category);

Would store a string like 'category/the-product-name'

Getting The Full Product URL

To get the full product URL, use the getProductUrl() function. For example:

$product->getProductURL();

Would store the full URL string like 'https://www.example.com/category/the-product-name'

Getting The Category URL Path

Getting the category URL in Magento is done using GetUrl() like this:

$category->getUrl();

Conclusion

Use this understanding of Magento's front and backend folder structure to keep your site optimized and outperform the competition. For more on your site's structure, take a look at our post on optimizing menus, navigation, and breadcrumbs in Magento, and feel free to check out our other posts about Magento SEO.