{"id":834,"date":"2009-02-10T19:59:56","date_gmt":"2009-02-10T18:59:56","guid":{"rendered":"http:\/\/www.jurecuhalev.com\/blog\/?p=834"},"modified":"2009-02-11T12:25:34","modified_gmt":"2009-02-11T11:25:34","slug":"hacking-and-headtracking","status":"publish","type":"post","link":"https:\/\/www.jurecuhalev.com\/blog\/hacking-and-headtracking\/","title":{"rendered":"Hacking and Headtracking"},"content":{"rendered":"<p>These are my notes on lecture Hacking and Headtracking by Domen Grabec that was part of <a href=\"http:\/\/pot.kiberpipa.org\">Cyberpipe&#8217;s open lectures<\/a>.<\/p>\n<p>The big question is &#8211; how can we make 3D worlds more accessible to end users, similiar to what <a class=\"zem_slink\" title=\"Johnny Lee (computer scientist)\" rel=\"wikipedia\" href=\"http:\/\/en.wikipedia.org\/wiki\/Johnny_Lee_%28computer_scientist%29\">Johnny Lee<\/a> did in his video:<\/p>\n<p><object width=\"425\" height=\"344\" data=\"http:\/\/www.youtube.com\/v\/Jd3-eiid-Uw&amp;hl=en&amp;fs=1\" type=\"application\/x-shockwave-flash\"><param name=\"allowFullScreen\" value=\"true\" \/><param name=\"allowscriptaccess\" value=\"always\" \/><param name=\"src\" value=\"http:\/\/www.youtube.com\/v\/Jd3-eiid-Uw&amp;hl=en&amp;fs=1\" \/><param name=\"allowfullscreen\" value=\"true\" \/><\/object><\/p>\n<p>So how can we achieve this without having to use WiiMote and to make it accessible to broader audience?<\/p>\n<p><strong>Headtracking 1.0<\/strong><\/p>\n<ul>\n<li><a class=\"zem_slink\" title=\"Wii Remote\" rel=\"wikipedia\" href=\"http:\/\/en.wikipedia.org\/wiki\/Wii_Remote\">Wii remote<\/a> library + Wii remote + headmounted LED&#8217;s<\/li>\n<li>Developing game<\/li>\n<li>Very robust solution<\/li>\n<li>Sadly not everyone has a Wiimote at home as well as that games have to use that library to use these features<\/li>\n<\/ul>\n<p><strong>Headtracking 0.6 or 1.4?<\/strong><\/p>\n<div class=\"zemanta-img zemanta-action-dragged\" style=\"margin: 1em; display: block;\">\n<div>\n<dl class=\"wp-caption alignright\">\n<dt class=\"wp-caption-dt\"><a href=\"http:\/\/www.flickr.com\/photos\/71184420@N00\/3270115112\"><img decoding=\"async\" title=\"Kiberpipa, Headtracking\" src=\"http:\/\/farm4.static.flickr.com\/3436\/3270115112_b98154ea4d_m.jpg\" alt=\"Kiberpipa, Headtracking\" \/><\/a><\/dt>\n<dd class=\"wp-caption-dd zemanta-img-attribution\" style=\"font-size: 0.8em;\">Image by <a href=\"http:\/\/www.flickr.com\/photos\/71184420@N00\/3270115112\">rok xxx<\/a> via Flickr<\/dd>\n<\/dl>\n<\/div>\n<\/div>\n<ul>\n<li>Why do we need to have these special glasses, if we can just hack games to use our library<\/li>\n<\/ul>\n<p><strong>Arhitecture<\/strong><\/p>\n<ul>\n<li>Application that tracks you head \ud83d\ude1b<\/li>\n<li>A compontent that injects itself into the game<\/li>\n<li>Communication library between these two things<\/li>\n<\/ul>\n<p>Second big question &#8211; <strong>How can we change the viewpoint of the camera<\/strong>?<\/p>\n<p>It turns out that the answer lies within <a class=\"zem_slink\" title=\"DirectX\" rel=\"homepage\" href=\"http:\/\/www.microsoft.com\/windows\/directx\/\">DirectX<\/a> library.<\/p>\n<p>How does DirectX rending work?<\/p>\n<ul>\n<li>World space &#8211; how the objects are distributed around the world<\/li>\n<li>View space &#8211; how camera views these objects<\/li>\n<li>Screen space &#8211; transforming this 3D space into 2D for monitor<\/li>\n<\/ul>\n<p>World space moves an object in local space to world space. It transforms the objects so we have a perception of depth and we have to do this for all the objects.<\/p>\n<p>View space moves world space to view space. Depending on where do we look at objects from, we need to move the camera around and change the calculations.<\/p>\n<p>Screen space once again moves view space into screen space \ud83d\ude42 Where we set near clip plane and far clip plane and visualize all the the objects in-between.<\/p>\n<p>\u00a0<\/p>\n<div class=\"zemanta-img zemanta-action-dragged\" style=\"margin: 1em; display: block;\">\n<div>\n<dl class=\"wp-caption alignright\" style=\"width: 250px;\">\n<dt class=\"wp-caption-dt\"><a href=\"http:\/\/www.flickr.com\/photos\/71184420@N00\/3270123900\"><img decoding=\"async\" title=\"headtracking2\" src=\"http:\/\/farm4.static.flickr.com\/3509\/3270123900_de3abcecc0_m.jpg\" alt=\"headtracking2\" \/><\/a><\/dt>\n<dd class=\"wp-caption-dd zemanta-img-attribution\" style=\"font-size: 0.8em;\">Image by <a href=\"http:\/\/www.flickr.com\/photos\/71184420@N00\/3270123900\">rok xxx<\/a> via Flickr<\/dd>\n<\/dl>\n<\/div>\n<\/div>\n<p><strong>How?<\/strong><\/p>\n<p>Two approaches:<\/p>\n<ul>\n<li>Start application in Debug mode<\/li>\n<li>Get table of function and their location in memory<\/li>\n<li>Override this function with our own implementation<\/li>\n<\/ul>\n<p>or<\/p>\n<ul>\n<li>ProxyDLL &#8211; we inject our <a class=\"zem_slink\" title=\"Dynamic-link library\" rel=\"wikipedia\" href=\"http:\/\/en.wikipedia.org\/wiki\/Dynamic-link_library\">DLL<\/a>, same as original<\/li>\n<li>We reroute all the functions to the original DLL<\/li>\n<li>We manipulate the values of the variables<\/li>\n<\/ul>\n<p>More about Proxy DLL<\/p>\n<ul>\n<li>All the exports of the original DLL need to be same in the proxy one<\/li>\n<li>Load the original DLL and reroute the functions<\/li>\n<li>Application has to locate ProxyDLL before original DLL<\/li>\n<li>We can spy after all the processes that use dynamic library loading<\/li>\n<\/ul>\n<p><a href=\"http:\/\/www.seeingmachines.com\/faceAPI.html\">Face API is really useful for this.<\/a><\/p>\n<p>\u00a0<\/p>\n<div class=\"zemanta-img zemanta-action-dragged\" style=\"margin: 1em; display: block;\">\n<div>\n<dl class=\"wp-caption alignright\" style=\"width: 250px;\">\n<dt class=\"wp-caption-dt\"><a href=\"http:\/\/www.flickr.com\/photos\/26119297@N00\/462291011\"><img loading=\"lazy\" decoding=\"async\" title=\"the viewport skews\" src=\"http:\/\/farm1.static.flickr.com\/244\/462291011_842ba06ce2_m.jpg\" alt=\"the viewport skews\" width=\"240\" height=\"218\" \/><\/a><\/dt>\n<dd class=\"wp-caption-dd zemanta-img-attribution\" style=\"font-size: 0.8em;\">Image by <a href=\"http:\/\/www.flickr.com\/photos\/26119297@N00\/462291011\">teafordinner<\/a> via Flickr<\/dd>\n<\/dl>\n<\/div>\n<\/div>\n<p>Interprocess communication<\/p>\n<ul>\n<li>Named pipes<\/li>\n<li>Similar usage as HTTP client\/server<\/li>\n<li>API for named pipes is the same as file system<\/li>\n<\/ul>\n<p>Limitations<\/p>\n<ul>\n<li>Lag of web cams<\/li>\n<li>Doesn&#8217;t work well in dark room<\/li>\n<li>Limitations of face recognition libraries<\/li>\n<li>It takes quite some CPU power<\/li>\n<li>You have to hack each game and it&#8217;s not simple if the library is not dynamically loaded<\/li>\n<\/ul>\n<p>We can work around most of the limitations with custom software<\/p>\n<p>\u00a0<\/p>\n<div class=\"zemanta-pixie\" style=\"margin-top: 10px; height: 15px;\"><img decoding=\"async\" class=\"zemanta-pixie-img\" style=\"border: medium none ; float: right;\" src=\"http:\/\/img.zemanta.com\/pixy.gif?x-id=016d1167-2992-40ea-9f33-54df55d02479\" alt=\"\" \/><\/div>\n","protected":false},"excerpt":{"rendered":"<p>These are my notes on lecture Hacking and Headtracking by Domen Grabec that was part of Cyberpipe&#8217;s open lectures. The big question is &#8211; how can we make 3D worlds more accessible to end users, similiar to what Johnny Lee did in his video: So how can we achieve this without having to use WiiMote [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[117],"tags":[685,454,684],"class_list":["post-834","post","type-post","status-publish","format-standard","hentry","category-conferences","tag-directx","tag-hacking","tag-headtracking"],"acf":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/posts\/834","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/comments?post=834"}],"version-history":[{"count":14,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/posts\/834\/revisions"}],"predecessor-version":[{"id":847,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/posts\/834\/revisions\/847"}],"wp:attachment":[{"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/media?parent=834"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/categories?post=834"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.jurecuhalev.com\/blog\/wp-json\/wp\/v2\/tags?post=834"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}