{"id":1768,"date":"2015-08-04T15:45:58","date_gmt":"2015-08-04T14:45:58","guid":{"rendered":"http:\/\/www.jurecuhalev.com\/blog\/?p=1768"},"modified":"2015-08-04T15:48:50","modified_gmt":"2015-08-04T14:48:50","slug":"wordpress-analytics-engine","status":"publish","type":"post","link":"https:\/\/www.jurecuhalev.com\/blog\/wordpress-analytics-engine\/","title":{"rendered":"WordPress Analytics Engine"},"content":{"rendered":"<p class=\"p1\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-1769\" src=\"https:\/\/www.jurecuhalev.com\/blog\/wp-content\/uploads\/2015\/08\/data-scheme-1024x667.png\" alt=\"data-scheme\" width=\"604\" height=\"393\" srcset=\"https:\/\/www.jurecuhalev.com\/blog\/wp-content\/uploads\/2015\/08\/data-scheme-1024x667.png 1024w, https:\/\/www.jurecuhalev.com\/blog\/wp-content\/uploads\/2015\/08\/data-scheme-550x358.png 550w, https:\/\/www.jurecuhalev.com\/blog\/wp-content\/uploads\/2015\/08\/data-scheme.png 1878w\" sizes=\"auto, (max-width: 604px) 100vw, 604px\" \/><\/p>\n<p class=\"p1\">I think I\u2019m obsessed with numbers. They give me a feeling of control. Page views, trends, visitor count and more. Not measuring things, makes me sad. If we have a historical data, we can check if our changes worked. Are certain topics more popular? Which stories are more popular on Twitter compared to Facebook? Infinite amount of questions.<\/p>\n<p class=\"p1\">I\u2019d like to build a better analytics engine for us. I\u2019ll explain my constraints, and how I\u2019d approach it. Primary plan is that someone can say &#8211; \u201cjust use X\u201d. If that fails, I can still build it.<\/p>\n<p class=\"p1\"><b>Problem definition<\/b><\/p>\n<p class=\"p1\">We have multiple WordPress installations with overlapping authors. They create blog posts, that are shared to Facebook and Twitter. Each post can include multiple embeds, that they produce &#8211; Sound Cloud, YouTube and similar. For each blog, we can also lookup into Google Analytics API and get stats on sessions, page views and time on site.<\/p>\n<p class=\"p1\">There are two primary limitations of these external data sources. Firstly, we\u2019re rate limited &#8211; so we can only query them about once a day &#8211; per post URL. Secondly, we mostly get aggregated data.<\/p>\n<p class=\"p1\">We would query these external API\u2019s about once a day. The only limitation we have is that we get aggregate\/sum data from them. Facebook only gives us total number of likes, so we need to make subtract previous value. This way we get number of likes in that day.<\/p>\n<p class=\"p1\"><b>Potential<\/b><\/p>\n<p class=\"p1\">Having all the information in one place, it would allow us a couple of things:<\/p>\n<ul class=\"ul1\">\n<li class=\"li3\">Weekly reports for authors &#8211; sending them encouragements on how their stories did<\/li>\n<li class=\"li3\">Information for content editors, what got most attention that week<\/li>\n<li class=\"li3\">Identify old content that suddenly got interest<\/li>\n<li class=\"li3\">Get information on success of embedded content (Sound Cloud, YouTube)<\/li>\n<li class=\"li3\">Develop customised indicators &#8211; authors with most viewed YouTube videos<\/li>\n<\/ul>\n<p class=\"p3\"><b>Potential Solution?<\/b><\/p>\n<p class=\"p3\">When researching this topic, there is a software stack that almost fits. It\u2019s <a href=\"https:\/\/www.elastic.co\/products\/logstash\">LogStash<\/a> with <a href=\"https:\/\/www.elastic.co\/products\/kibana\">Kibana<\/a>. LogStash provides data storage and logging capabilities. Kibana support display of data in many different ways.<\/p>\n<p class=\"p3\">The other approach would be to just code it any web framework. But it seems like a huge duplication of work.<\/p>\n<p class=\"p3\"><b>Technical Questions<\/b><\/p>\n<p class=\"p3\">Would ELK stack work? Can LogStash provide input filter that will automatically normalise data for me? It is the right technology stack at all?<\/p>\n<p class=\"p3\">Is there anything that solves this in a much\u00a0better way?<\/p>\n<p class=\"p3\"><b>Content Questions<\/b><\/p>\n<p class=\"p3\">Is it worth building this at all? Is it a good idea to attach numbers to (journalists) work? Did I miss any questions that would be worth exploring?<\/p>\n<p class=\"p3\">Would you use such a service?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I think I\u2019m obsessed with numbers. They give me a feeling of control. Page views, trends, visitor count and more. Not measuring things, makes me sad. If we have a historical data, we can check if our changes worked. Are certain topics more popular? Which stories are more popular on Twitter compared to Facebook? Infinite [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[14],"tags":[],"class_list":["post-1768","post","type-post","status-publish","format-standard","hentry","category-tech"],"acf":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/posts\/1768","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/comments?post=1768"}],"version-history":[{"count":3,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/posts\/1768\/revisions"}],"predecessor-version":[{"id":1772,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/posts\/1768\/revisions\/1772"}],"wp:attachment":[{"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/media?parent=1768"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/categories?post=1768"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/tags?post=1768"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}