How’dy!
We’ve took one step toward personalization!
Some of you have asked us how to customize dashboard, here it is now!
Open your dashboard, and move widgets around. put them wherever you prefer them to be.
You can even resize that big active visitors widget and make it smaller!
If you think our current default layout of dashboard widgets is not appealing, share yours with us at sales@hitsteps.com
We would like to see it from your side 🙂
There is an issue we faced while part of our database to another server far far away for performance and stability. However, SCP appears to transfer at 2MB/s where network was 10MB/s. we cancelled transfer and used alternatives.
What are alternatives you can use?
First, you can speed up SCP by turning on compression which will have a slight effect:
scp -C -o 'IPQoS throughput' -o 'CompressionLevel 9' -c arcfour file
however, there seems to be some issue when transfer time passed 5 minutes, it just get slow and slower due to network latency. Rsync was not an option as we needed a parallel way.
If your both servers are in same data center, then either rsync or scp would do just fine, but in our case, We came up with Axel! it is not quite similar to others, but does job perfectly. It works excellent when data centers are far away.
Axel is a parallel HTTP downloader which support resume, and can download as fast as available bandwidth. Transfer time has been cut down to 30 minutes and 10MB/s (fastest our network support)
A simple command can such as:
axel -a -v -n 8 http://domain.com/somerandomfolder/largefile.zip
can do the job and use maximum capacity of your internet. However be aware that once you copy a file in your HTTP address it will be accessible to the world, make sure URL is kept secret and you delete file from URL as soon as download is finished.
You might need to install axel via
yum install axel apt-get install axel
We had a scenario here, which we needed to distribute load between few servers, buying a load balancer was not an option (or at least we wanted to experiment with DNS Load balancing) and it worked great.
Afterall, A load balancer is just another point of break in case your load balancer crash!
Please note that this load balancing won’t check status of servers and randomly return IP addresses (but your visitor won’t see a crashed server!)
Another side effect is that DNS load balancer won’t keep same visitor on same server on each page refresh or link change, so you need a shared session for all servers.
PHP’s default session uses file system and isn’t made for sharing between servers, so you’ll need a shared session manager. memcached is a great option here. sync all your server PHP sessions by connecting them to memcached of a server.
Next step, you need to make sure your database is replicated as master-master and file system is synced (you can use lsyncd for file system sync)
If you have more than 2 servers, you might need to use MySQL 5.7 above, because they offer multi-source master replication, so your slave server can connect to 2 masters at same time.
Before MySQL 5.7, multi-master replication was done using round robin replication which causes big problem in case one of servers go offline.
Now, lets see what will happen when you add multiple A records to your DNS.
so, lets say we have this record:
A www.hitsteps.com 123.123.123.123
We change it to:
A www.hitsteps.com 123.123.123.123 A www.hitsteps.com 123.123.123.124
Where 123.123.123.123 is your first server’s IP address and 123.123.123.124 is your second server IP address.
So here is what happens in our experience. Browser randomly choose one of these IP addresses. since file system and session and database is cached, visitor will always see correct page regardless of which server he is connected to.
It is really random and we cannot
Now, if one of your servers crash and IP address become unavailable, browser will choose another available IP address, therefor your visitors won’t see any downtime. Once crashed server come back online, browser will start serving pages using this server again.
Update: Starting 9th November 2016, Cloudflare will no longer support API v1 which we’ve used in this post. You may look for migrating to v4 and example scripts such as lyoshenka’s great work 😉
Howver you can read our explaination below to figure out how we achieve Failover using CloudFlare service
We use Cloudflare for our service and we use it to have server switchover or failover in case our main server go down, and it been working perfectly ever since!
Wouldn’t it be perfect to have a DNS failover, saving costs by avoiding a hardware and also reduce risk of having main load balancer go down? It is possible to have DNS Load balancing by having two IP Addresses point to same record A, but we go for it in another article.
In this article we will cover how to have CloudFlare perform automatic IP failover for your servers, so in case main server go down, backup server kick in.
Please note for this to work seamlessly, you need your backup server be exactly same as main server. You can achieve this by having MySQL replicated and file system synced using lsyncd. You can also have your backup server show a temporarily message.
If you were to use a real DNS to set IP, it would take hours for new IP to apply to DNS servers around globe, and your visitors would still get pointed to crashed server but not with Cloudflare. new DNS will kick in less than few minutes (1min approx, and max 5mins)
You will need to run monit server in your backup server, ideally you can run monit on a separate server which only have monit server on it. You can get a simple and budget friendly vps linux cloud server from vpsdime and have it check status of your main server.
Please get your API key from Cloudflare. You can get it from Account tab in cloudflare. Cloudflare is free!
Then you need to shell scripts, one to change IP to backup, and one to change IP to main. Here we have provided both scripts for you. You can download them HERE and set appropriate configs into both files (IP addresses, cloudflare key, email address…)
You need to upload files in your backup server, in this example we use folder /hitsteps/cloudflare/
type in ssh console:
mkdir /hitsteps/ mkdir /hitsteps/cloudflare/
and then upload configured files there.
Make sure you give execute permission to this two files:
chmod +x /hitsteps/cloudflare/*.sh
Now, you need to install monit in your backup server:
yum install monit (or apt-get install monit)
and configure your monit to check your main server status by editing /etc/monit or /etc/monitrc and add following lines to end of it, edit 0.0.0.0 with your main server IP address and change alert email address to your address:
check host 0.0.0.0 with address 0.0.0.0 alert your@email.com if failed icmp type echo count 3 with timeout 3 seconds for 2 cycles then exec "/bin/bash -c /hitsteps/cloudflare/tobackup.sh" else if succeeded for 2 cycles then exec "/bin/bash -c /hitsteps/cloudflare/tomain.sh" if failed url http://0.0.0.0/ with timeout 20 seconds and retry 2 for 2 cycles then exec "/bin/bash -c /hitsteps/cloudflare/tobackup.sh" #else if succeeded for 2 cycles then exec "/bin/bash -c /hitsteps/cloudflare/tomain.sh"
Now you need to restart monit, and make sure it run on startup:
service monit restart chkconfig monit on
Now, monit will check status of your main server by pinging it, if ping failed for 3 times on 3 cycles, then it will fire tobackup.sh file, same will happen if it cannot access webserver for 2 times.
Once server is accessible, it will run tomain.sh.
tobackup.sh will connect to cloudflare API and change your domain IP addresses to backup server. this change usually apply in less than 1 minute by our experience. monit will keep monitor your main server status and change IP back once it is accessible again.
As a sidenote, your cloudflare folder which contain scripts should have write permission because it will write logs into it. you also need to have following apps installed in your linux box for tomain.sh and tobackup.sh parse cloudflare API correctly: bc, cut, curl, sed, head, python
yum install python bc cut curl sed head
Do you have any suggestion to enhance this script? or any feedback? Please do let us know!
SSD make difference!
We’ve been hearing that SSD hard disks are new generation and therefor, all database intensive servers have to move to new SSD servers. We gave it a shot and we never regret it!
In hitsteps, we analyze thousands of hits per seconds. this hits then categorized and processed by a background processor, and archived somewhere along with millions of other hits!
Given searching a visitor or looking back into archive of a visitor is an essential feature of hitsteps, we always had to struggle with performance of database searches, specially when searching for some visitors who visited months ago.
MySQL databases need RAM to hold all indexes of data, they also need a fast disk in case data are not in RAM. so we did the test in a mirrored server:
Our search parameter was a simple query of “Searching for a visitor for who visited hitsteps dashboard who are from Canada since 6 months ago”
HDD server took 45 seconds to return query (almost near to timeout in most apache web server setups). SSD server took only 6 seconds to return query. Going to page number 2 in HDD took 15 seconds, in SSD just a wee 2 seconds.
It worth mentioning all other simple queries take less than 0.2 seconds in SSD servers and 0.5 in HDD servers.
Please do note that this times are closed source and relative to our script and codes. It might be different with other Applications, codes and scripts you might use in your server. RAM also play a major role here which is more appearance if your server is using HDD.
Bonjour!
It’s been a while we are working on this one, french language is hard to learn, and we now realized, it is also hard to translate! Took us 1 month and half to finally get language ready all over hitsteps, but it come just on time for this new year, we welcome 2016 with a new language!
If you are a french user and you have selected french as your language during signup, hitsteps now automatically show french language to you,
New users will be detected by their geolocation and correct language will be set for them, so everything goes smooth and automatic.
If you would like to get back to english language, you can do so by using language switcher at footer of our homepage.
There are lot of languages out there, but it take very long time to translate for each of them, therefor we will plan a community driven system where everybody can collaborate their own language and shape hitsteps for their own, wait for that!
We are happy to announce a new feature, called “Cross-Device” or “Visitor Merging”, you get the idea.
You can see full customer journey in your website. So your users can access your services on their phones, tablets and desktop. Hitsteps now will keep all of their different devices profile in one unique profile, making it easy for you to keep track of each particular users.
Basically, what it do is to assign an Unique ID to your visitors (say their email) once they leave a comment or once they login.
Next time same user come to your website, even with another device, mobile or tablet, once they login back, Hitsteps will detect their previous profile and merge them together. so that new session will be continue of user’s previous session. be 1 day before or 1 month before.
You can register your hitsteps account now and give it a try. If you use wordpress, be sure to use our WordPress plugin to automatically integrate this feature.
It comes handy when your users register on your website from mobile device, then few days later continue their browsing from a Desktop and purchase something from you. If profiles were to kept separated, you look at profile who purchases from you and you have no idea how this visitor even find your website! Worry no more!
Our WordPress users can update their WordPress plugin to v4.81 now and benefit from this feature on their WordPress based website (WooCommerce and such)
Here is how it looks like in your visitor profile reporting:
Our other users will need a little bit of coding.
Just put following code above hitstep’s tracking script, you need to define a unique ID using server-side scripting, and we will take care of rest:
<script>
_hs_uniqueid=”YOUR_DEFINED_UNIQUE_ID”;
</script>
so that you can define a unique ID for each user, say their email address or their User ID and have it linked to visitor who is browsing your website.
While you are there, don’t forget you can set a name for your visitor using ipname parameter as well:
<script>
ipname=”YOUR_VISITOR_NAME”;
</script>
Once a user login and If we have user’s unique ID stored before, All events he fired and pages he viewed perior to login, will be linked to main profile ID by Hitsteps Analytics and make a unified timeline of full user behaviour.
Since we have launched Hitsteps Shopify App last week, we have applied many minor changes to perfect it.
It is a deal breaker for your business and make you be able to understand your visitors in deeper depth.
Here is how to install Histeps Web Analytics on your Shopify:
We are proud to announce hitsteps full integration with Shopify ecommerce and online shop service.
If you have a shopify shop, don’t worry about changing themes and inject hitsteps code. Simply install hitsteps app from the Shopify App Store.
The app will automatically install, add your website to hitsteps, create predefined shopify labels for you to track sales and products, and automatically install hitsteps code on all pages of your Shopify Store.
Today we have a real deal for you, for those of you who are tired of getting excited by a high peak on your traffic only to know that it’s been a Microsoft Bingbot, Google bot, Facebook crawler or any other bot crawling around in your website.
We had a basic bot filtering in place since our launch last year, but now we have tighten this system.
Now we actively scan each one of your visitors to make sure we are only providing you with real human analytics who are accessing your website.
It helps to give you a real analytics insight, purely based on real human visitors and avoid fake generated pageviews from various web services.
For example, if somebody share your page on facebook, facebook will open your website to take a snapshot for thumbnail photos. this process will repeat for each new shares. Normally you might think you are getting lot of visitors, but in fact they are all Facebook thumbnail generator bots trying to open your website and get list of photos to show them upon sharing. It give you false hope and screw your analytics data. We have eliminated it!
As a result of filtering this visitors from your analytics, Hitsteps might show slightly lower hits compared to other analytics services where they count all hits. It come to our attention that many many analytics still track visitors like in ancient days and ignoring the fact that nowadays, many web services have a tool to scan your website and thus, they show you not so useful analytics insight.
If you were still unsure about hitsteps analytics, but now you made up your mind, go ahead! We will meet you inside. Get Hitsteps WordPress plugin.
