Category Archives: Meaningful work

Maja Rihar on basics of office ergonomics for developers

In Meaningful work interviews I talk to people about their area of work and expertise to better understand what they do and why it matters to them.

Maja Rihar (LinkedIn) is an ergonomics consultant specializing in a workplace environment. In our conversation, I tried to learn current best practices in how to set up my work environment.

What do you do?

I teach people how to best use ergonomic accessories for office use. I have past experience with managing offices and I was also in charge of ensuring workplace safety. While the popular belief is that it’s not dangerous to work in an office my experience and statistics disagree with this. That’s why I’ve started helping people to adjust workspaces to each individual worker.

Why are office workers getting hurt in the long run?

What I’ve noticed is that corporate training on workplace safety of employees is not effective. There’s also an issue that our population is getting taller on average – about 12 cm in the last 100 years. Our national office equipment standards do not adjust fast enough to these population changes.

Our current standards are made for people that are on average 178 cm in height. If you’re over 180 cm in height you’ll have problems with working in your current office environment. Lots of tall people are telling me about their back problems.

Any tips on what common mistakes you see among IT workers in their offices?

One very common mistake I see is that people have their keyboards too far from their bodies. It should be about 20 cm from the body. During a workshop, we’ll measure this distance and make sure that you have a visual feeling for where it should be.

I also suggest that you set an alarm for every 15 minutes that reminds you to stand up and sit down with a correct posture. Once you sit down with a correct posture you’ll once again be able to sit correctly for a while. Anything longer than about 15 minutes makes the body slouch so 15 minutes is a good time interval to move your body. Just doing this would prevent a lot of long-term damage to the musculoskeletal system.

Laptops shouldn’t be used for any long-term work. They are ergonomically correctly set up only when used with an external keyboard with a screen on a raised platform. The constant hunched-down position is very hard on the body. This is further compounded by all the mobile and other device use.

I’m happy to see more and more standing desks in offices. I think this is one of the best things that you can do for your body. Mainly because it forces you throughout the day to keep moving all the time as you switch from standing to sitting setup and back.

How do you pick good equipment?

I think it’s important that you trial many different vendors and that you are picky with what you choose. Don’t just try one office chair. Try many, figure out if you’ll know how to adjust it correctly, and also ask your peers what worked for them.

When setting up a home office environment you can learn from your work office. If you ever try to work from your kitchen table you’ll quickly get a feeling if it’s a better or worse experience than your office table. This can help you guide your decisions and you’ll know how to improve it toward what feels good.

It’s also a good idea to talk to experts before making these choices. Any office equipment that you’ll buy will probably be used for many hours every day for the next ten years. Getting good advice upfront can be a good investment in such a context.

Any more practical advice?

You should think about how you work. Make sure to adjust your body and how you work so that it will support you in a long run. 

Invest in quality lighting. Get into the habit that you’re opening windows and adding fresh air to your space.

Take breaks often and move your body during these breaks. I don’t think it’s healthy to work for long periods and then go train for another long period. It’s much better to mix both throughout the day.

Talk to your peers about how they’re solving their problems. Seeing how others adjust their workspace makes it much easier for you to imagine how you could improve yours. The more embodied your experience with trying different accessories is the easier it will be to get a feel if it’s right for you.

Can you recommend any additional resources on this topic?

Official guidance

Vendors

Good YouTube Channels on this topic:

What I learned from talking with Maja

The embodied experience of trying different ergonomic accessories is worth the effort.

Using external triggers to make sure that I check on my pose.

Standing desk provides a lot more benefits than I expected and the hype seems justified.

Sara Vošinek Gašpar explains why developers should switch jobs every 3 years

In Meaningful work interviews I talk to people about their area of work and expertise to better understand what they do and why it matters to them.

Sara Vošinek Gašpar is an IT recruiter. She explained to me why job ads in the IT industry are so strange, the specifics of the Slovenian IT market, and how developers can further level up. After talking with her I started to see the benefits of building a relationship with a recruiter and not just randomly applying to job ads.

How do others introduce you?

They would say that I’m an IT recruiter and a talent sourcer. I would add to that that I see myself more as a talent enabler. I’m always trying to figure out how to empower developers and find opportunities where they can grow.

What is it like to work with developers for you?

Sometimes I hear from people that it’s weird to work with developers. I think that’s a lazy excuse. My experience is that developers are super chill and very straightforward people. The nature of their job is creative and they want to protect their creativity. They want to be very quickly on the same page, very informal, and straight to the point. They’re very quick to notice red flags and potential issues with clients that try to recruit them.

This makes my job very easy as I don’t need to waste time and very hard as it makes it very hard for me to recruit people for certain companies that have a reputation. 

That doesn’t mean that it’s impossible to find developers that would work in such an organization. It just takes more work to find someone that is driven by the idea that they can fix the culture, make such systems work more reliably, or just like the mission of the company.

Developer job ads often mention the perk of having access to fresh fruit and coffee machines in the office. Is that really a factor for developers when considering new employment?

Yes, it turns out that it’s a perk for some people. There is a non-trivial amount of companies that still don’t offer this and you need to buy or bring your own coffee.

What I see, at least in Slovenia, is that developers have much better working environments compared to other positions with a similar level of education. Job ads are traditionally written by the HR department and for them, these things are genuine perks that they probably don’t even have access to. There’s also the difference in the salary (developers usually earn a lot more compared to HR) and they’re constantly recruited by the competition. Bringing all of these differences together sometimes results in job ads that list benefits that are self-evident to developers.

What can we developers do to make job ads more realistic?

If developers compare their jobs to non-developers jobs they’ll always feel like they have a really great working environment, perks, and pay. The trick is that they need to compare their job to other developers’ jobs. This is the best way to see if you’re actually in a bad job and that you need to switch for something better.

There’s still a lot of opportunities for developers to push back on bad hiring and working practices. At the moment there’s still too much power in the hands of the companies . Developers are the only ones digitally literate and it will remain so  for the next 15-20 years so it is crucial  for developers to  take back creative control. We need  to support open-source and new R&D developments while protecting our data and empower developers to be the counterweight to other interests in the digital market.

How can we help improve the working environment for developers?

Mostly by educating them on what’s possible. Presenting them with alternative approaches to professional growth, team organization, or work practices.

I really like when companies allow developers to work on non-work projects during work time so that they can familiarise themselves with new technologies, for  example Internet of Things, drones, playing around with Raspberry Pi or Arduino,…. It could also be free time that they can use to fix whatever issue they find annoying in their existing codebase or learn something new.

I also wish more developers would talk about how they use SCRUM or Agile methodologies and actually have Scrum Masters in their team. To explain to other developers how to push back on bad implementations from management. That you shouldn’t remove retrospectives because someone arbitrarily decided that they’re not needed.

What are some specifics that are unique to the IT market in Slovenia?

In Slovenia, it’s possible to survive as an IT company just by virtue that your software has a Slovenian language interface. These kinds of companies can exist for 15+ years with about 30 clients that keep them afloat. They are very risk-averse and don’t really want to change or update any technologies or processes. It also means that the whole company is stagnating. Developers don’t learn anymore and salaries are low when compared to the market. It would be good for the market and the developers if some of these companies would go out of business. They’re holding the whole market back.

How often should a developer switch jobs?

If it’s your first job you need to absolutely switch if you’re there more than 3 or 4 years. It doesn’t matter if it’s a good team and you enjoy your work. You need to switch so that you’ll learn something new and see how different teams operate.

I see people that are technologically obsolete after their first two years at their job. They got assigned such a specific dead-end technological job that they don’t have any experience with modern IT practices. It’s much harder to help them with finding their next job as they don’t have any real modern development experience. This should not happen to a young person and they need to watch out for it.

Job hopping is the best way to achieve exponential growth of your potential. You get to see how other people think and do things. Having multiple experiences help you understand what you want and with that knowledge, it’s much easier to find a job where you’ll stay longer (5 – 10 years).

What’s the profile of a developer that you’re most looking for?

I’m looking for someone with 10 – 15 years of work experience with modern technologies. They already switched their job a few times and have a range of experience of working in different teams. They understand technology enough to know when the latest hyped technology is not a good fit for their project. This profile is in its prime at the moment.

How do you comment on general burnout feeling among developers and often stated wish to exit industry “before they are too old to code”?

I don’t understand developers that say that they can’t imagine still coding when they’re in their 60’s. I can’t imagine hearing this from a painter. That they can’t imagine painting anymore when they’re 60. I can understand that a painter doesn’t want to work on commissioned art anymore. But they still want to paint!

I’m wondering how and when they lose this appreciation for solving technological problems. Writing code is just a medium for how you solve these challenges and not the end goal.

What tends to happen is that they are put in a position where they’re unhappy. Fixing bugs, working with technology that they hate, deadlines, and an uninspiring team. After a few years of this, they mentally burn out and want to exit the industry. Developers need to notice this early enough and switch their job.

A developer’s job should be all about learning new mental models, how to solve new types of problems, and to keep growing.

How should (non-junior) developers market themselves so that they get access to better job opportunities?

You should have your basic profiles set up: LinkedIn, StackOverflow, and GitHub. That will help you get the first round of opportunities.

After that, you should look into niche communities around Reddit, Hacker News, Slack, and Discord. That’s where founders and senior technical staff hang out. If you can meaningfully participate in these communities you’ll be able to connect with thought leaders in the space. These will be the same people that can then hire you or connect you to opportunities that are not even advertised yet.

You need to think about how you would hire a skilled developer. Where would you go to meet them and how would you identify them? Then do that yourself so that other developers will notice you.

Are there any resources that you’d recommend for leveling up?

What I learned from talking with Sara

Switching jobs every few years is now expected. Not switching jobs often enough can be a potential liability.

There’s a quiet tension between the creative work of developers and managers that try to control processes around it. It’s probably worth exploring more.

There’s still a lot of opportunities to create support communities for developers and related profiles.

Jana Bergant teaches me how to get started with creating paid online courses

In Meaningful work interviews I talk to people about their area of work and expertise to better understand what they do and why it matters to them.

Jana Bergant (LinkedIn, YouTube, Udemy) is a developer, creator, and author of online courses with over 25000 students. She kindly answered my questions on how to approach creating commercial online courses. We’ve also discussed personal growth for developers and how to find your space in the world.

How do you introduce yourself to people?

There are three main perspectives on my professional work at the moment. I’m running online courses so I’m an instructor to my students and I’m there to support their learning. For my consulting business, I’m an online elearning coach to my clients and I help them build their own courses. In this context, I’m helping them with a mindset, technology, content structure and I support them on their e-learning journey. For occasional contract work, I’m also an e-learning expert that implements educational resources for my clients. My last project was API documentation and project documentation for a slovenian company BetterCare (Marand).

I’m someone that follows my passions: technology, writing, consulting, and teaching. 

How did you get to the point where you are now – a successful online teacher?

I first got involved in programming during my studies at the Faculty of Organizational Sciences in Kranj. It really clicked for me and I started developing accounting software for my father’s store. After this experience, I applied for an internship at six different local companies and immediately got offers from four. My pitch was “I’m new to programming, passionate about the topic, and these are the things that I would like to learn” and it worked. For a while, I’ve also tried studying Computer Science in addition to finishing my first studies and also working two jobs. It was too much and I ended up just working and learning things as I needed them for my work. So, yes, I’m a self-taught developer. I’ve done a variety of work, mostly as a freelancer.

A couple of years ago I had a burnout, got an autoimmune disease that made my life really hard, then got pregnant and I broke up with my partner. It was a really hard time for me. Raising a small child alone made it hard for me to continue with a busy freelancing lifestyle. I wanted to have more time and attention for my daughter and to reduce the overall stress in my life. This is how I found SmartNinja and joined them as an instructor. That’s where I discovered Udemy and made my first course just as an experiment. It was for Jekyll, a static website generator. I enjoyed the process so I then created a second course where I tried to teach Javascript in a more fun and relaxed way. As later chatbots became popular I found them interesting and I created a course on Facebook Messenger Chatbots. Each step was an organic continuation of all that I learned in the past.

I’m now starting to focus more on connecting and growing my consulting practice as making online courses alone at home can be quite lonely at times.

You mentioned that you did some freelancing work in the past. Do you have any advice for freelancers?

Yes, don’t get stuck in a routine. As a freelancer, you should know your niche and at the same also be a generalist that connects things together. It’s good if you can bring in other fields. I’ve also found that it’s important that you’re the kind of person that is not afraid to show yourself to the world. I discovered this difference myself when I started recording my courses. They gave me more exposure to different audiences and as a consequence, I got opportunities that I wouldn’t otherwise have.

If I could do things again I would start speaking, networking, writing books, and doing online courses much much earlier. It requires you to get out of your comfort zone but the upsides are worth it.

What drives you to leave your comfort zone?

I need to see meaning in my work. If there’s none I just can’t do it for long, and just going to work every day doesn’t do it for me.

I’ve also discovered as I became a mother that I want to be an example for my daughter. I want her to be feminine, caring, and empathic and at the same time also to be courageous and full of self-worth. This wish for her is coming from my experience growing as a person.

I used to do a lot of work for free and it took me a while to learn that I need to start charging realistic rates for my work. I don’t subscribe anymore to the idea that in code development things should be free. 

What did you learn from making your first online course?

Students must get a feeling of accomplishment very early on. In the case of Udemy, where there are a lot of very low price courses, students often buy many courses at the same time. You need to provide a strong hook for students so that they will engage with your content and actually learn. On Udemy students can also ask questions to the course authors. You should address these questions daily. It’s not a lot of work though, about 10 – 15 minutes per day.

Reflecting on the course itself also taught me how I could structure it better and what would make it even more useful. Putting each step into a software version control (GIT) also makes it easier for students to follow along. 

At the moment I see these courses also as a paid advertisement for my work. Someone pays to see how I work and think. This generated a lot of interesting freelance work in the past with clients from all over the world. It’s unlikely that you’ll get rich from making a course on Udemy. Some do though 🙂

What does your process for creating an online course look like?

I start with market research. Tools like moz.com and AnswerThePublic give me a general idea of the amount of demand for the topic. If you’re already an expert in your field you can also investigate what kind of more niche content is missing in your area of work. 

What I’m seeing is that you need to go beyond just creating a course. You’ll need to create a community that will have courses, books, and other activities as part of it. If you just want to do a course on Udemy there are specific strategies around that but it will still require understanding the people that you’re addressing with your course.

The bigger the niche the easier it will be for you to address your community. For example:

  • Lisa K created a community where students get together and practice intuition to drive better decisions.
  • Kate Olson trains service dogs.

I’m also not limiting myself to one specific platform for the delivery of online courses. Some niches love real-time Zoom meetings while others want to have discussion forums and self-paced learning. It’s more important to go into a really specific niche such as “yoga for women that are going into menopause” instead of just offering general yoga courses. It will allow you to use her language and to connect with her feelings and needs through your offerings. It’s the same thing also if you’re trying to create something for developers. You need to find a specific niche (digital agencies, beginners, ..) and what specific pain are you helping them with.

After I’ve identified my niche customer I then try to figure out their current behavior patterns. What do they search for on Google and which websites do they visit on my topic? That gives me a list of websites or communities that I need to start engaging to get in touch with them and maybe offer them whatever I’m offering.

This all sounds like a lot of work and you haven’t even started to explain the practical parts of building a course.

That’s true. Building something like this requires a very different mindset from working as a freelancer or having a job. It’s not a fixed scope project where you know how much work it will be and what you are going to get paid. Trading time for money of course makes a good living lifestyle but doesn’t generate passive income streams. 

To create revenue streams that ‘passively’ generate the income you need to go out of your comfort zone and take risks. I think that here in Slovenia we’re very risk-averse. 

My first course wasn’t very successful and I could decide to go back to a day job. In my view, in life you fail many times and it’s a part of the process to learn something new.

When you are creating courses, the topic should also be something that you’re enthusiastic and passionate about. If you’re doing courses only for the money it won’t work.

How do you decide on which projects to focus your time on?

That’s definitely a challenge that I still struggle with. It does become easier with experience. I’ve learned it the hard way in the past when I said yes too many times and that led to health problems in my life. I would really advise everyone to find their limits early and not discover them the hard way. 

Do you use any special equipment when producing your video courses?

I have two extra LED lights, I record myself with my phone and I use a Rode microphone. For screen recording and editing I use Camtasia. There’s no need for anything more complicated.

How long does it take you to produce your videos all together (content, scripting, recording, and editing)?

For my YouTube channel, it’s about one day of work for one ten-minute video. For an Udemy course where you need much more content, it can be easily two months of work. It’s probably possible to do it faster but that’s not stopping me at the moment from doing the work.

I recognize that it’s risky in terms of required time investment and that there are no guarantees that the videos will sell at all. It is still a necessary step if you want to build additional revenue streams.

What are some of your favorite resources for leveling up and people that you learn from?

I would suggest reading at least one book per month that helps you grow. It’s an investment you’ll never regret. I also have coaches I hire to help me get better in different areas. If you can move faster by learning from someone who has already solved what you are struggling with, why not use it and learn from it. Not all lessons need to be learned the hard way!

What I learned from talking with Jana

Technological aspects of creating online courses are the easy part. Mindset and being willing to invest time into the long-term is the hard part.

It’s a journey and it takes many tries to arrive at the point where an outsider can see success.

It’s quite possible to decide on what kind of life you want and then adjust the type of work that you do to that.

Larsen Cundrič shares how to level up during studies

In Meaningful work interviews I talk to people about their area of work and expertise to better understand what they do and why it matters to them.

Larsen Cundrič is in his final year of undergraduate Computer Science studies. We also talked about his previous entrepreneurial experience. At the time of this conversation, he was just finishing his last week of exams during his Erasmus exchange in Denmark.

What’s your current focus?

I’m deep into learning how to be a data scientist. How to organize data, build pipelines, and how that connects to creating prediction models. I’m already working with an early-stage biotechnology startup so all of this is not just theoretical. I’ve also had previous apprenticeship experience in creating prediction models in an ad tech company.

How do you currently see the role of a data scientist?

From what I’ve seen so far, this role requires a very diverse set of skills. You need to understand a lot of statistics and how to do data processing. It also requires you to know how to visualize all of this data.

You mentioned that you’re looking to specialize in Machine Learning and Artificial Intelligence. Why did you decide on it and not for example Web or Mobile development?

Before I started with my studies I was competing with my team as a part of the FIRST LEGO League. The task was to program the robot so it would autonomously solve the challenges in the competition. That’s when I knew I wanted to be in the field of programming.

As I started my studies I couldn’t connect with Web or Mobile development. I enjoyed mathematics and logical thinking much more. In my second year of studies, I stumbled upon a Data Science Bootcamp on Udemy. That’s where I discovered that data science is a perfect field of work for me. It’s a mix of math and computer science and you still get to implement practical solutions in innovative ways. I enjoy the complexity of connecting so many different disciplines together.

Are there any outcomes of your technology that fascinates you at the moment?

My work in a biotechnology startup feels really magical to me at the moment. We’ve developed a process using data science approaches that let us analyze your blood sample and deduct your age from it. That seemed like science fiction to me when I started my studies and now I’m part of the team that is developing such technology.

It’s also amazing how fast it’s possible to learn all of this. It only took me 3½ years to be able to work in data science, develop Android applications, and many more things. I also didn’t learn just the technology but also about the engineering aspects of projects.

You were also active as an entrepreneur. What did you learn from those attempts?

I’ve had two previous entrepreneurial projects. We created a brand of beeswax cosmetics, and we were developing a concept of gamification in marketing.

What I’ve learned from these experiences is that there are many more options in life than just having a job. You can build a company. You can do project-based work. You can join a startup and try to change the world. Concepts and opportunities that were completely foreign to me before.

I’ve also learned a lot from launching new business projects. I think the main lessons were more about the mechanism of running a business. How to keep track of finances, setting goals, and how to divide and delegate responsibilities. It also requires much more attention and focus than I expected. I’ve also learned that I’m currently more interested in technology so I’ve shifted my attention away from business development.

Overall I’ve discovered that the general opinion of what’s hard to achieve doesn’t always apply to me. I’ve always heard that it’s hard to study math and computer science. So it’s a good thing that I tried it for myself and discovered that it isn’t that hard. So now I know that I need to experience things for myself to be able to know if it’s really hard or not.

Can you recommend any good resources to level up?

What I learned from talking with Larsen

A good approach to learn about oneself is to give things try and evaluate if it’s a good fit or not.

The best students are supplementing their studies with self-directed online learning.

With every failure there’s a lot of learning that comes out from it.

Matej Martinc explains Natural Language Processing

In Meaningful work interviews I talk to people about their area of work and expertise to better understand what they do and why it matters to them.

Matej Martinc is a Ph.D. researcher at “Jožef Stefan” Institute in the Department of Knowledge Technologies where he invents new approaches on how to work and analyze written text. He explained to me the basics of Natural Language Processing (NLP), why neural networks are amazing, and how one gets started with all of this. In the second half, he shared how he ended up in Computer Science with a Philosophy degree and why working for companies like Google is not something that interests him.

How do people introduce you?

They introduce me as a researcher at the IJS institute. I’m in the last year of my Ph.D. thesis research. I’m mostly working on Natural Language Processing (NLP). NLP is a big field and I’m currently exploring several different areas.

I initially started by automatically profiling text authors by their style of writing – we can detect their age, gender, and psychological properties. I also worked on automatic identification of text readability. We’ve also created a system to detect Alzheimer’s patients based on their writing.

Lately, I’ve been working on automatic keyword extraction and detecting political bias in word usage in media articles. I’m also contributing to research on semantic change – how word usage changes through time.

References to research that Matej is referencing throughout this interview. I encourage you to read them as they’re written in a very clear language.

Scalable and Interpretable Semantic Change Detection

[..] We propose a novel scalable method for word usage change detection that offers large gains in processing time and significant memory savings while offering the same interpretability and better performance than unscalable methods. We demonstrate the applicability of the proposed method by analyzing a large corpus of news articles about COVID-19

Zero-Shot Learning for Cross-Lingual News Sentiment Classification

In this paper, we address the task of zero-shot cross-lingual news sentiment classification. Given the annotated dataset of positive, neutral, and negative news in Slovene, the aim is to develop a news classification system that assigns the sentiment category not only to Slovene news, but to news in another language without any training data required. [..]

Automatic sentiment and viewpoint analysis of Slovenian news corpus on the topic of LGBTIQ+

We conduct automatic sentiment and viewpoint analysis of the newly created Slovenian news corpus containing articles related to the topic of LGBTIQ+ by employing the state-of the-art news sentiment classifier and a system for semantic change detection. The focus is on the differences in reporting between quality news media with long tradition and news media with financial and political connections to SDS, a Slovene right-wing political party. The results suggest that political affiliation of the media can affect the sentiment distribution of articles and the framing of specific LGBTIQ+ specific topics, such as same-sex marriage.

Can you start by explaining some background about NLP (Natural Language Processing) to start with?

As a first step, it’s good to consider how SVM (support vector machine) classifiers and decision tree techniques used for classification work. Very broadly speaking, they operate on a set of manually crafted features extracted from the dataset that you train your model on. Examples of that type of features would be: “number of words in a document” or a “bag of words model” where you put all the words into “a bag” and a classifier learns which words from this bag appear in different documents. If you have a dataset of documents, for which you know into which class they belong to (e.g., a class can be a gender of the author that wrote a specific document), you can train your model on this dataset and then use this model to classify new documents based on how similar these documents are to the ones in the dataset on which the model was trained. The limitation of this approach is that these statistical features do not really  take semantic relation between words into account, since they are based on simple frequency-based statistics.

About 10 years ago a different approach was invented using neural networks. What neural networks allow you to do is to work with unstructured datasets because you don’t need to define these features (i.e., classification rules) in advance. You train them by inputing sequences of words and the network learns on itself how often a given word appears closer to another word in a sequence. The information on each word is gathered  in a special layer of this neural network, called an embedding layer that is basically a vector representation that encodes how a specific word relates to other words. 

What’s interesting is that synonyms have a very similar vector representation. This allows you to extract relations between words. 

An example of that would be trying to answer: “Paris in relation to France” is the same as “Berlin in relation to (what?)”. To solve this question you can take the embedding of Paris, subtract the embedding of France and add embedding of Berlin and you’ll get an embedding as an answer – Germany. This was a big revolution in the field as it allows us to operationalize relations in the context of languages. The second revolution came when they invented transfer learning, a procedure employed for example in  the BERT neural network that was trained on BookCorpus with 800 million words and English Wikipedia with 2500 million words. 

In this procedure, the first thing you want to do is to train a language model. You want the model to predict the next word in a given sequence of words. You can also mask words in a given text and train the neural network to fill the gaps with the correct words. What implicitly happens in such training is that the neural network will learn about semantic relations between words. So if you’re doing this on a large corpus of texts (like billions of words in BERT) you get a model that you can use on a wide variety of general tasks. Because nobody had to label the data to do the training it means that it’s an unsupervised model.

Are you working with special pre-trained datasets?

I’m now mostly working with unsupervised methods similar to the BERT model. So what we do is to take that kind of model and do additional fine-tuning on a smaller training set  that makes it better suited for that specific research. This approach allowed us to do all of the research that I’m referencing here.

A different research area that doesn’t require additional training is to  employ clustering on the embeddings of these neural networks. You can take a corpus of text from the 1960s and another one from the 2000s. We can then compare how usage of specific embeddings (words) compare between these two collections of texts. That’s essentially how we can study how the semantic meaning of words changed in our culture.

Modern neural networks can also produce embedding for each usage of a word, meaning that words with more than one meaning have more than one embedding. This allows you to differentiate between Apple (software company) and apple (fruit). We used this approach when studying how different words connected to  COVID changed through time. We generated embeddings for each word appearance in the corpus of news about COVID and clustered these word occurrences into distinct word usages. Two interesting terms that we identified were diamond and strain. For strain, you can see the shift from using it in epidemiological terms (strain virus) to a more economic usage in later months (financial strain).

What we showed with our research is that you can detect changes even across short (monthly) time periods. There’s a limit to how accurately we can identify the difference. It’s often hard even for humans to decide how to label such data. We can usually get close to humane performance by using our unsupervised methods.

(both figures are from paper Scalable and Interpretable Semantic Change Detection)

Does this work for Non-English languages?

You can use the same technology with a non-English language and we’re successfully using it with Slovenian language. In the case of  viewpoint analysis of Slovenian news reporting, we’ve discovered a difference in how the word deep is used in  different context. Mostly because of the deep state that became a popular topic in certain publications.

For our LGBTIQ+ research, we can show that certain media avoids using the word marriage in the context of LGBTIQ+ reporting and replaces it with terms like  domestic partnership. They’re also not  discussing LGBTIQ+ relationship within the context of terms such as family. We can detect the political leaning of the media based on how they write about these topics.

We just started with this research on the Slovenian language so we expect that we’ll have much more to show later in the year.

(figure is from paper Automatic sentiment and viewpoint analysis of Slovenian news corpus on the topic of LGBTIQ+)

So far you’ve talked about analysis and understanding of texts. What other research are you doing?

We’re working on models for generating texts as part of the Embeddia project. The output of this research also works with the Slovenian language.

We’re also investigating if we can transfer embeddings between languages. We have a special version of the BERT neural network that has been trained on 100+ different language Wikipedias. What we’ve found out is that you can take a corpus of texts in the English language, train the model on  it to, for example, detect the gender of the author, and then use that same model to predict the gender of the author of some Slovenian text. This approach is called a zero-shot transfer.

How approachable is all this research and knowledge? Do I need a Ph.D. to be able to understand and use your research?

It takes students of our graduate school about a year to become productive in this field. The biggest initial hurdle is that you need to learn how to work with neural networks.

Good thing is that we now have very approachable libraries in this field. I’m a big fan of PyTorch as it’s well integrated with the Python ecosystem. There’s also TensorFlow that’s more popular in the industry and less in research. I found it harder to use for the type of work we’re doing and harder to debug. With PyTorch it takes about a month or two for our students to understand the basics.

In our context, it’s not just about using the existing neural networks and methods. Understanding the science part of our field and how to contribute via independent paper writing and publishing it’s usually about 2 years.

How easy is it to use your research in ‘real-world’ applications?

We have some international media companies that are using our research in the area of automatic keyword extraction from text. We’re helping them with additional tweaking of our models.

Overall we try to publish everything that we do under open access licenses with code and datasets publicly available.

What we don’t do is maintain our work in terms of production code. It’s beyond the scope of research and we don’t have funding to do it. It’s also very time-consuming and it doesn’t help us with our future research. That’s also what I like about scientific research. We get to invent things and we don’t need to maintain and integrate them. We can shift our focus to the next research question.

So in practice, all of our research is available to you but you’ll need to do the engineering work to integrate it with your product.

Let’s shift a bit to your story and how you got into this research. How did you get here?

I first graduated in philosophy and sociology in 2011, at the time when Slovenia was still recovering from the financial crisis. While I considered Ph.D. in philosophy I decided that there are not many jobs for philosophers. That’s why I’ve enrolled in a Computer Science degree that offered better job prospects.

During my Computer Science studies, I was also working in different IT startups. I quickly realized that you don’t have a lot of freedom in such an environment. Software engineering was too constrained for me in terms of what kind of work I could do.

After I graduated I took the opportunity to do Erasmus Exchange and I went to University in Spain. In that academic environment, I found the opposite approach. I received a dataset, a very loose description of a problem, and complete freedom to decide on how I’m going to approach and solve the problem.

When I returned to Slovenia I decided to apply to a few different laboratories inside IJS to see if I could continue with academic research. I’ve got a few offers and accepted the offer from the laboratory where I’m working today. 

I also decided to focus on NLP and language technologies as I’m still interested in doing philosophical and sociological research. Currently, I have the freedom to explore these topics in my research field without too many constraints. I’m also really enjoying all the conferences and travel that comes with it. Due to the fast-changing nature of my field, all the cutting-edge research is presented at conferences, and publishing in journals is just too slow. It takes over a year to publish a paper but there’s groundbreaking research almost monthly.

How do you see research done at FAANG (Facebook, Amazon, Apple, Netflix, Google) companies? We know that they’re investing a large amount of money into this field and have large research teams.

They’re doing a lot of good research. At the same time, they’re also often relying more on having access to a large number of hardware resources that we don’t. This can be both a blessing and a curse. At the moment I don’t see their research being that much better from the findings from universities. Universities are also more incentivized to develop new optimization techniques as they can’t use brute hardware force for their research.

Are you considering working for a FAANG company after your Ph.D.?

Not really. I already have a lot of freedom in my research and I can get funding to explore the areas that interest me. If I would work inside a FAANG company I would need to start at the bottom of the hierarchy and also be limited by their research agenda.

I also really like living in Slovenia and I don’t want to relocate to another country. At the same time, I’m excited about potential study/researchexchanges as I enjoy collaborating with researchers at foreign institutions.

What are some good resources to follow in your field?

You can follow the current state of the art at:

Papers describing paradigm shifts in the field of NLP:

Unsupervised language model pretraining and transfer learning: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

What I learned from talking with Matej

  • Recognizing what kind of work makes you happy allows you to optimize your job or clients so that you do such work.
  • Natural Language Processing is a very approachable technology and not something that only big companies can use.
  • There are many opportunities to bring research findings into the industry. It does require expertise and connections to both fields.
  • These technologies now also work for the Slovenian language.