Oncrawl

OnCrawl Review – SEO Crawler And Log Analyzer In-Depth

For any business even remotely interested in improving visibility and ranking, the dream is to get the inside scoop on Google in order to get ahead.

The holy grail of the internet, getting to number one on Google – or at least close to it – is the one thing that keeps business owners up at night.  So, how’s it done?

You could do this by placing an undercover operative into the internet giant’s hallowed offices in order to gain intel, however, there is an easier way.

There are a huge number of tools out there which promise to get inside Google’s head (so to speak) and improve your SEO and, my aim is to road-test these tools – deal? 

The tool for this article is award winner, OnCrawl.

OnCrawl is an SEO platform that provides technical data and analysis to help over one thousand enterprise companies in 66 countries improve their organic traffic. OnCrawl was recognized as the most awarded SEO platform in 2021 and gives search marketers easy access to actionable dashboards and reports.

What the heck is OnCrawl?

OnCrawl is a web-based, data access automation software focusing on technical SEO solutions to help businesses better understand which informations are important and, how they can use it to improve positions, organic traffic, ranking and search engine results. 

Similarly, OnCrawl helps brands to identify HTML errors as well as issues with linking structure and content – all the nasties that can send a brand into a nose-dive in Google’s ranking.

Truly global, OnCrawl has offices in Bordeaux and Montreal and has users from all around the world climbing onboard. 

Who is OnCrawl for?

OnCrawl is for any individual or business who needs to keep at least one eye on how their website is performing – and how that performance is affecting its Google ranking. 

From freelancers relying on their site for contracts to huge corporations likewise, OnCrawl’s wide range of features means that it’s a great tool for anyone who’s on the world wide web. 

Who are OnCrawls customers?

Testimony to OnCrawl’s success is the large number (over 1000 and counting) of businesses who are already using it. 

Prominent companies such as Forbes, Lastminute.com, TicketMaster and iProspect – and many more have published glowing testimonies as to the effectiveness of the tool – which means that you’ll be in good company if you decide to go with OnCrawl.  

On Crawl customers
OnCrawl customers

How does OnCrawl work?

OnCrawl is web-based so there’s no need to mess about with fiddly software.  To begin with, OnCrawl offers a free 14 day trial (no credit card needed), which I urge you to take advantage of.  Start by getting those fingers clicking over to https://app.oncrawl.com/signup and fill in a super-short registration form (Name, phone and email address – as mentioned, no credit card details needed).  

On Crawl: Signup form
OnCrawl: Signup form

Next, once you’ve handed over your details, you’ll be asked to pick a package (details on pricing to follow, I promise).  After you’ve got this locked down, you’ll be asked to verify your shiny new account by SMS.  

OnCrawl Trial
OnCrawl Trial

Crawl before you run

It doesn’t take long to get the official stuff out of the way and get on with exploring your new OnCrawl account.  Once you’re in, it works like this: 

First, you’ll be invited to watch a short tutorial which will tell you everything you need to know about inputting your projects into OnCrawl.  This ten minute video is incredibly helpful in getting you on your way. 

On Crawl: Advice and answers from the OnCrawl Team
OnCrawl: Advice and answers from the OnCrawl Team

Secondly, after your projects are loaded, you’ll be directed to a page that looks something like this: 

On Crawl: Tutorials
OnCrawl: Tutorials

And lastly, from here, you can click onto any one of your projects to view results.  Tap onto the project you want and scroll down until you see ‘Crawl Report’ and then, you guessed it, click onto that. 

On Crawl: Dashboard
OnCrawl: Dashboard

You’ll now be presented with a summary tab which is, basically, an accumulation of all your important crawl data information. 

This will tell you all you need to know, including

  • how many pages have been crawled
  • indexed or no-index and
  • pesky errors that may be harming your ranking
Oncrawl pages by groups
OnCrawl Segmentation

When you tap onto a particular chart, you can then get your hands on the Data Explorer which will give you even more detail, should you need it; including info about the title, status code and meta robots. 

On Crawl: Data Explorer
OnCrawl: Data Explorer

OnCrawl’s easy navigation means that you can add new filters whenever you feel the urge.  

Navigate back to your summary and you can access information including:

  • Load time
  • Links breakdown
  • HTML quality
  • Status code errors

Don’t worry if you get stuck, there are plenty of help options available, including live chat. 

Crawling the ranks

By now, I’ve given you a pretty good idea of what OnCrawl does in basic terms and, how to get started for your site or business. 

Now, it’s time to delve a little deeper into what OnCrawl does and, how it does it. As you now know, OnCrawl is all about creeping round your site (and your competitor’s) to spot issues and, help you to fix them.

To help users better understand what’s going on on their sites, OnCrawl uses two unique rank tools, InRank and FreshRank.

Inrank

Think of Inrank as an internal page ranking tool – and one which is vital for the health of a site. 

Developed by OnCrawl’s team, Inrank shows the popularity of each of your pages within the internal link architecture by allocating a score from 0 to 10.

OnCrawl's Inrank
OnCrawl’s Inrank

The scoring

Every time it performs a crawl, OnCrawl calculates the Inrank of each page it comes across. 

The first page it encounters will generally be the Home Page or start page and, Inrank will usually assign an Inrank of 10 to this page, depending on the crawl settings. 

Next, as OnCrawl continues its journey around the site, subsequent pages will be scored depending on the number and quality of inlinks (an external hyperlink to a webpage) pointing to the page. 

Other factors which may affect the scoring include the quality of the anchor text and content evaluations.

Within this latter, particular note will be taken of duplicate content and overall quality.  

Why Inrank?

Above all, this metric is a really convenient way of measuring the success (or not) of your internal linking strategy – particularly if your objective is to improve your ranking. 

Inrank takes into account a number of important factors such as backlinks, content and the loading speed of pages – all of which can have a significant impact on your rank. 

For example, the number of inlinks pointing to a particular page can affect the way that a search engine analyses that page and, therefore, the overall authority of the page and the site. 

Using Inrank to your advantage

To begin with, Inrank is connected to the crawl size scale and, so, you should always be running full reports for all of your pages in order to get the full picture and to benefit from this tool. 

Although it sounds like stating the obvious, crawling a limited section of your site will consequently result in limited data results on your internal link structure. 

To clarify, Inrank scoring is derived from internal links, it can be used to understand how certain kinds of optimisation impacts the popularity of your pages. 

Hence, this allows you to avoid common mistakes such as lowering the authority of your whole site by attempting to improve the authority of multiple pages at once. 

Inrank will also direct you to optimising the pages that need it the most – such as your top product or brand pages.

When looking at your Inrank score, a general rule of thumb is that your depth to depth Inrank ratio should be no higher than two. 

Using Inrank to beat the competition

When ranking websites, Google applies a number of parameters to a number of websites which match the query. 

Fresh Rank

The second weapon in OnCrawl’s ranking arsenal is Fresh Rank.  Again, just like Inrank, Fresh Rank was created by the clever folk at OnCrawl and is a unique and exclusive tool for OnCrawl users.  

Oncrawl Fresh Rank
Fresh Rank

All about the SEO, Fresh Rank measures the time taken between a page’s first ever Google crawl to its first SEO visit.  By doing this, the tool is able to show users how many days, on average, have elapsed between these two important events.

Why Fresh Rank?

The time lapse between the first ever search engine crawl and the first ever SEO visit is important as it speaks to the visibility and reputation of your website as a whole. 

The Fresh Rank is important as it gives a quick indication of this visibility and reputation by applying scoring.

For example, a website with great visibility and important authority will have a low scoring Fresh Rank.  On the other hand, a smaller website with inferior visibility will have a higher Fresh Rank.

This tool allows you to see immediately where your site fits into this score.

Using Fresh Rank to your advantage

To get the most out of Fresh Rank, there are a few things to take into consideration: 

Check your SEO

In order for Fresh Rank to be effective, all new pages published on your site need to be sufficiently optimised for SEO. 

As I’ve explained, Fresh Rank works by measuring the time between a search engine crawl and an SEO visit so, if your page isn’t optimised correctly, the results will reflect this. 

Content, content, content

In case you haven’t heard, content is everything when it comes to your online presence. 

Make sure that Fresh Rank has plenty to sink its teeth into by making sure that your pages are jam packed with fresh, unique, engaging and SEO optimised content. 

Netlinking is the term used to describe a hyperlink which points to your page from a third party website and is one of the fundamental factors used by Fresh Rank. 

For this reason, it’s important that your netlinking is plentiful and high quality to improve the visibility of your site.

OnCrawl’s Features

OnCrawl has three main fabulous features which are:

  • SEO Crawler
  • Log Analyser
  • SEO Impact Report

I’ll tell you a little more about these:

SEO Crawler

The main attraction for OnCrawl is, of course, its SEO Crawler.  This is the feature which is responsible for monitoring your pages and content. 

The SEO Crawler provides valuable insights which enable you to improve the quality of your pages and content – including deleting any duplicate content.

In turn, this allows you to provide the best user experience possible. 

OnCrawl: Types of problematic duplicate content per page
OnCrawl: Types of problematic duplicate content per page  
OnCrawl: Site-wide duplicate content types
OnCrawl: Site-wide duplicate content types

Log Analyser

This clever little tool allows you to examine your log files in order to see how others, including bots and Google, are interacting with your site. The Log Analyser will also show you which status codes are being returned – for example, the dreaded ‘404’ so that you can take action quickly.

Data3

Data3 a.k.a SEO Impact Report is a combination of the crawl and log files as well as additional data from Google Analytics or Google Search Console.

The tool puts these three together to give you an overview report of what’s happening with your page structures. This will also tell you what Google is crawling. 

How it works for a user

As a user, this multifaceted tool can be a godsend. Here’s the rundown of how it works: 

Define your project

By project, we generally mean a website, set of websites or a domain which share Search Console and Analytics Accounts – this is the kind of stuff that OnCrawl will analyse.

Connect your data sources

Essentially, OnCrawl pulls data from internal and external sources in order to form a full picture of your site. This data will include: 

  • Data collected by OnCrawl: SEO crawl results, server log analysis and data scraped from web pages.
  • Data collected by other sources: Ranking data, backlinks data and analytics data. This also includes data from tools like SEMrush or Ahrefs.
OnCrawl: Connect your data sources
OnCrawl: Connect your data sources

Set your crawl parameters

OnCrawl’s clever bot can be customised – so you can set limits in the number of URLs or more in depth within the site architecture. You can also choose to override robots.txt files, crawl subdomains and manage cookies, HTTP headers, sitemaps and crawl speed.

Launch or schedule the analysis

Once you’ve got your ducks in a row in terms of data and have set up your crawl, you’re ready to get your analysis underway. You can choose to launch your analysis immediately or schedule it for later. 

Scheduling Crawls
OnCrawl: Scheduling Crawls

Email alerts

Once your crawl and analysis is complete, OnCrawl will ping you an email with an overview of the results.

Reports

Post-analysis, you can view comprehensive dashboard reports covering over 500 charts 😅 in order to help you to understand the results.  These include log monitoring, crawl results and cross analysis. 

Segment and search data

Now, the real fun starts.  You can segment your data in order to see specific areas, such as: 

  • Number of conversions
  • Number of impressions
  • Click through rate
  • Ad spend
  • Word count
  • Publication date
  • Blog posts & InRank

You’ll also have access to the data behind any of the charts. The tool comes with present data filters but you can choose to write your own and add or remove columns to create more precise results. You can also export your results to keep or share.  

OnCrawl: Segmentation
OnCrawl: Segmentation
Blog posts segmentation
OnCrawl: Blog posts segmentation

OnCrawl’s Pricing

OnCrawl is available in three refreshingly simple packages: 

Explorer Package starting at $69/mo

The smallest package is perfect for a limited budged and in-house SEO. The features are:

  • 100 000 URLs to crawl per month
  • 1 project (domain)
  • Unlimited users
  • Unlimited exports (CSV, JSON, Pdf)
  • Online support

Business Package starting at $249/mo

The Business package has a bit more power and the features include:

  • 500 000 URLs to crawl per month
  • 2 projects (domains)
  • Unlimited users
  • Unlimited exports (CSV, JSON, Pdf)
  • Online support

Infinite & Beyond Package – Price on request

The last package can be personalized to meet the needs of large SEO agencies and enterprise websites. Here are the features:

  • Custom URL quota & projects
  • Custom log lines volume
  • Javascript crawling
  • Custom data sources
  • Data Studio connector
  • Big Data exports & API access
  • VIP Support & Onboarding

To see the detailed list of features and pricing, make sure to visit this page.

Pros and Cons of OnCrawl

A review wouldn’t be a review without the all-important Pros and Cons.  Here are the ups and downs of using OnCrawl. 

The Ups

  • Easy peasy graphics – OnCrawl’s results are displayed in a really clear and common sense way, with simple graphics and graphs.  These simple to understand graphics make OnCrawl really straight-forward to use, even for non-technical people. 
  • Features – With the three main features, OnCrawl is an all in one tool for those looking for results in a simple to use format. 
  • Speed – OnCrawl is able to crawl unlimited URLs depending on the package chosen – and it does this at a whopping 300 URLs per second, making it one of the fastest crawl tools on the market.
Crawling speed
OnCrawl: Crawling speed
  • GDPR – Has full GDPR compliance so users can rest assured that they’ll stay on the right side of the law. 
  • Expert Support – OnCrawl’s support team is made up of experienced SEOs and engineers – so you’ll be put in touch with someone who knows what they’re talking about!
  • Huge reach – Can support sites which have literally millions of pages.
  • Custom reporting – All plans include custom reporting which is incredibly important for many businesses.   

The Downs

  • Price – It is a little more expensive than other tools, although, to be honest, this is reflected in its performance.  
  • Keyword monitoring – Unfortunately the tool doesn’t currently include keyword research or monitoring so users will need to use a different tool for this. 
  • In advance – At times users may need to contact support when tackling some advanced options. 
  • On alert – As of yet, Oncrawl’s full alerting features are not in place (although I am informed that they’re on their way). 

Are there tools similar to OnCrawl?

JetOctopus

A new kid on the block that allows you to crawl your websites without any subscription. You just buy crawl credits on demand, which sets a new precedent for website crawling and analysis. You don’t need to pay monthly, just pay whenever you need the service. It’s a tool that offers you all the features of OnCrawl but with a revolutionary payment model.

DeepCrawl

DeepCrawl is probably the biggest competition for OnCrawl. They’re pretty similar, even name-wise. It offers basically the same thing – in-depth SEO page audits, technical insights, segmentation, data management and analytics.

SEMRush

SEMRush is a suite that offers all of the tools needed to get your page optimized the right way. Apart from technical SEO guidelines, it enables you to do page audits, research keywords, create optimized content, analyze the SERP and much more.

Ahrefs

Ahrefs is another all-in-one SEO platform. You don’t need to be an SEO pro to use Ahrefs to your advantage. It provides everything you would ever need to optimize your page both technically and content-wise.

Conclusion

It’s kind of difficult to find fault with the tool as it really does do what it says on the tin. OnCrawl’s speed and reliability are great, as is the easy to use interface and navigation.

If I were to have a complaint at all, it would be that the basic package really is, well, basic; a few more features here would be nice. All in all though, OnCrawl is a great tool for businesses of all sizes. 

Oncrawl

As seen on

Featured posts

Popular posts

More posts

Monitoling baclinks

Monitor Backlinks Review – Find your Good & Bad Links

You already know how important backlinks are for your business website (and, if you don’t then, quite frankly, you have no business having a business …

read more

Surfer SEO Review 2023 – Boost Your Website Ranking

Looking to up your SEO game? Surfer SEO might just be the answer. This powerful tool provides a range of features that can help boost …

read more

Don’t buy SurferSEO until you read this – Surfer SEO Review 2024

Looking to up your SEO game? Surfer SEO might just be the answer. This powerful tool provides a range of features that can help boost …

read more

Cooperate with: