Jana Bergant teaches me how to get started with creating paid online courses

In Meaningful work interviews I talk to people about their area of work and expertise to better understand what they do and why it matters to them.

Jana Bergant (LinkedIn, YouTube, Udemy) is a developer, creator, and author of online courses with over 25000 students. She kindly answered my questions on how to approach creating commercial online courses. We’ve also discussed personal growth for developers and how to find your space in the world.

How do you introduce yourself to people?

There are three main perspectives on my professional work at the moment. I’m running online courses so I’m an instructor to my students and I’m there to support their learning. For my consulting business, I’m an online elearning coach to my clients and I help them build their own courses. In this context, I’m helping them with a mindset, technology, content structure and I support them on their e-learning journey. For occasional contract work, I’m also an e-learning expert that implements educational resources for my clients. My last project was API documentation and project documentation for a slovenian company BetterCare (Marand).

I’m someone that follows my passions: technology, writing, consulting, and teaching. 

How did you get to the point where you are now – a successful online teacher?

I first got involved in programming during my studies at the Faculty of Organizational Sciences in Kranj. It really clicked for me and I started developing accounting software for my father’s store. After this experience, I applied for an internship at six different local companies and immediately got offers from four. My pitch was “I’m new to programming, passionate about the topic, and these are the things that I would like to learn” and it worked. For a while, I’ve also tried studying Computer Science in addition to finishing my first studies and also working two jobs. It was too much and I ended up just working and learning things as I needed them for my work. So, yes, I’m a self-taught developer. I’ve done a variety of work, mostly as a freelancer.

A couple of years ago I had a burnout, got an autoimmune disease that made my life really hard, then got pregnant and I broke up with my partner. It was a really hard time for me. Raising a small child alone made it hard for me to continue with a busy freelancing lifestyle. I wanted to have more time and attention for my daughter and to reduce the overall stress in my life. This is how I found SmartNinja and joined them as an instructor. That’s where I discovered Udemy and made my first course just as an experiment. It was for Jekyll, a static website generator. I enjoyed the process so I then created a second course where I tried to teach Javascript in a more fun and relaxed way. As later chatbots became popular I found them interesting and I created a course on Facebook Messenger Chatbots. Each step was an organic continuation of all that I learned in the past.

I’m now starting to focus more on connecting and growing my consulting practice as making online courses alone at home can be quite lonely at times.

You mentioned that you did some freelancing work in the past. Do you have any advice for freelancers?

Yes, don’t get stuck in a routine. As a freelancer, you should know your niche and at the same also be a generalist that connects things together. It’s good if you can bring in other fields. I’ve also found that it’s important that you’re the kind of person that is not afraid to show yourself to the world. I discovered this difference myself when I started recording my courses. They gave me more exposure to different audiences and as a consequence, I got opportunities that I wouldn’t otherwise have.

If I could do things again I would start speaking, networking, writing books, and doing online courses much much earlier. It requires you to get out of your comfort zone but the upsides are worth it.

What drives you to leave your comfort zone?

I need to see meaning in my work. If there’s none I just can’t do it for long, and just going to work every day doesn’t do it for me.

I’ve also discovered as I became a mother that I want to be an example for my daughter. I want her to be feminine, caring, and empathic and at the same time also to be courageous and full of self-worth. This wish for her is coming from my experience growing as a person.

I used to do a lot of work for free and it took me a while to learn that I need to start charging realistic rates for my work. I don’t subscribe anymore to the idea that in code development things should be free. 

What did you learn from making your first online course?

Students must get a feeling of accomplishment very early on. In the case of Udemy, where there are a lot of very low price courses, students often buy many courses at the same time. You need to provide a strong hook for students so that they will engage with your content and actually learn. On Udemy students can also ask questions to the course authors. You should address these questions daily. It’s not a lot of work though, about 10 – 15 minutes per day.

Reflecting on the course itself also taught me how I could structure it better and what would make it even more useful. Putting each step into a software version control (GIT) also makes it easier for students to follow along. 

At the moment I see these courses also as a paid advertisement for my work. Someone pays to see how I work and think. This generated a lot of interesting freelance work in the past with clients from all over the world. It’s unlikely that you’ll get rich from making a course on Udemy. Some do though 🙂

What does your process for creating an online course look like?

I start with market research. Tools like moz.com and AnswerThePublic give me a general idea of the amount of demand for the topic. If you’re already an expert in your field you can also investigate what kind of more niche content is missing in your area of work. 

What I’m seeing is that you need to go beyond just creating a course. You’ll need to create a community that will have courses, books, and other activities as part of it. If you just want to do a course on Udemy there are specific strategies around that but it will still require understanding the people that you’re addressing with your course.

The bigger the niche the easier it will be for you to address your community. For example:

  • Lisa K created a community where students get together and practice intuition to drive better decisions.
  • Kate Olson trains service dogs.

I’m also not limiting myself to one specific platform for the delivery of online courses. Some niches love real-time Zoom meetings while others want to have discussion forums and self-paced learning. It’s more important to go into a really specific niche such as “yoga for women that are going into menopause” instead of just offering general yoga courses. It will allow you to use her language and to connect with her feelings and needs through your offerings. It’s the same thing also if you’re trying to create something for developers. You need to find a specific niche (digital agencies, beginners, ..) and what specific pain are you helping them with.

After I’ve identified my niche customer I then try to figure out their current behavior patterns. What do they search for on Google and which websites do they visit on my topic? That gives me a list of websites or communities that I need to start engaging to get in touch with them and maybe offer them whatever I’m offering.

This all sounds like a lot of work and you haven’t even started to explain the practical parts of building a course.

That’s true. Building something like this requires a very different mindset from working as a freelancer or having a job. It’s not a fixed scope project where you know how much work it will be and what you are going to get paid. Trading time for money of course makes a good living lifestyle but doesn’t generate passive income streams. 

To create revenue streams that ‘passively’ generate the income you need to go out of your comfort zone and take risks. I think that here in Slovenia we’re very risk-averse. 

My first course wasn’t very successful and I could decide to go back to a day job. In my view, in life you fail many times and it’s a part of the process to learn something new.

When you are creating courses, the topic should also be something that you’re enthusiastic and passionate about. If you’re doing courses only for the money it won’t work.

How do you decide on which projects to focus your time on?

That’s definitely a challenge that I still struggle with. It does become easier with experience. I’ve learned it the hard way in the past when I said yes too many times and that led to health problems in my life. I would really advise everyone to find their limits early and not discover them the hard way. 

Do you use any special equipment when producing your video courses?

I have two extra LED lights, I record myself with my phone and I use a Rode microphone. For screen recording and editing I use Camtasia. There’s no need for anything more complicated.

How long does it take you to produce your videos all together (content, scripting, recording, and editing)?

For my YouTube channel, it’s about one day of work for one ten-minute video. For an Udemy course where you need much more content, it can be easily two months of work. It’s probably possible to do it faster but that’s not stopping me at the moment from doing the work.

I recognize that it’s risky in terms of required time investment and that there are no guarantees that the videos will sell at all. It is still a necessary step if you want to build additional revenue streams.

What are some of your favorite resources for leveling up and people that you learn from?

I would suggest reading at least one book per month that helps you grow. It’s an investment you’ll never regret. I also have coaches I hire to help me get better in different areas. If you can move faster by learning from someone who has already solved what you are struggling with, why not use it and learn from it. Not all lessons need to be learned the hard way!

What I learned from talking with Jana

Technological aspects of creating online courses are the easy part. Mindset and being willing to invest time into the long-term is the hard part.

It’s a journey and it takes many tries to arrive at the point where an outsider can see success.

It’s quite possible to decide on what kind of life you want and then adjust the type of work that you do to that.

Larsen Cundrič shares how to level up during studies

In Meaningful work interviews I talk to people about their area of work and expertise to better understand what they do and why it matters to them.

Larsen Cundrič is in his final year of undergraduate Computer Science studies. We also talked about his previous entrepreneurial experience. At the time of this conversation, he was just finishing his last week of exams during his Erasmus exchange in Denmark.

What’s your current focus?

I’m deep into learning how to be a data scientist. How to organize data, build pipelines, and how that connects to creating prediction models. I’m already working with an early-stage biotechnology startup so all of this is not just theoretical. I’ve also had previous apprenticeship experience in creating prediction models in an ad tech company.

How do you currently see the role of a data scientist?

From what I’ve seen so far, this role requires a very diverse set of skills. You need to understand a lot of statistics and how to do data processing. It also requires you to know how to visualize all of this data.

You mentioned that you’re looking to specialize in Machine Learning and Artificial Intelligence. Why did you decide on it and not for example Web or Mobile development?

Before I started with my studies I was competing with my team as a part of the FIRST LEGO League. The task was to program the robot so it would autonomously solve the challenges in the competition. That’s when I knew I wanted to be in the field of programming.

As I started my studies I couldn’t connect with Web or Mobile development. I enjoyed mathematics and logical thinking much more. In my second year of studies, I stumbled upon a Data Science Bootcamp on Udemy. That’s where I discovered that data science is a perfect field of work for me. It’s a mix of math and computer science and you still get to implement practical solutions in innovative ways. I enjoy the complexity of connecting so many different disciplines together.

Are there any outcomes of your technology that fascinates you at the moment?

My work in a biotechnology startup feels really magical to me at the moment. We’ve developed a process using data science approaches that let us analyze your blood sample and deduct your age from it. That seemed like science fiction to me when I started my studies and now I’m part of the team that is developing such technology.

It’s also amazing how fast it’s possible to learn all of this. It only took me 3½ years to be able to work in data science, develop Android applications, and many more things. I also didn’t learn just the technology but also about the engineering aspects of projects.

You were also active as an entrepreneur. What did you learn from those attempts?

I’ve had two previous entrepreneurial projects. We created a brand of beeswax cosmetics, and we were developing a concept of gamification in marketing.

What I’ve learned from these experiences is that there are many more options in life than just having a job. You can build a company. You can do project-based work. You can join a startup and try to change the world. Concepts and opportunities that were completely foreign to me before.

I’ve also learned a lot from launching new business projects. I think the main lessons were more about the mechanism of running a business. How to keep track of finances, setting goals, and how to divide and delegate responsibilities. It also requires much more attention and focus than I expected. I’ve also learned that I’m currently more interested in technology so I’ve shifted my attention away from business development.

Overall I’ve discovered that the general opinion of what’s hard to achieve doesn’t always apply to me. I’ve always heard that it’s hard to study math and computer science. So it’s a good thing that I tried it for myself and discovered that it isn’t that hard. So now I know that I need to experience things for myself to be able to know if it’s really hard or not.

Can you recommend any good resources to level up?

What I learned from talking with Larsen

A good approach to learn about oneself is to give things try and evaluate if it’s a good fit or not.

The best students are supplementing their studies with self-directed online learning.

With every failure there’s a lot of learning that comes out from it.

First spike for Roam to WP plugin

Status update on getting the first “spike/prototype” of code working.

Scaffolding initial plugin code

I’ve looked into a number of scaffolding solutions:

I’ve initially tried to use WordPress Plugin Boilerplate Powered but it generated so much boilerplate code that it was just too much work to actually write my own code.

So I ended up using wp scaffold that produced a very nice skeleton that I could start writing code into.

Settings Screen

I first needed to build a screen where I could upload my exported data. Final result looks like this for now:

What was really useful in this research was article by Delicious Brains – 5 Ways to Create a WordPress Plugin Settings Page. I ended up using Carbon Fields as I didn’t want to invest too much time into scaffolding my own HTML Form code just to process a few fields.

Processing the data export

For now the general idea is that I only need to look at the first level of blocks inside Roam. So I’ll have structure like this:

{
  "create-time": 1621153344466,
  "title": "!Resources/BASB/Reviews",
..
"children": [
  {
  "string": "Wordpress:: #publish",
  ..
  ":create/user": {
    ":user/uid": "D9GErGIgWMcqL1SJ9egralKACQ62"
  },
  ..
},
..
]
}

and I need to find all blocks that have somewhere in the first level of children a string that matches "Wordpress:: #publish". The problem is that tree searching libraries like jsonq only return the matching child node but not the parent structure.

No problem we just need to write a single node depth search:

<?php 
function extract_nodes_to_publish( array $data ): array {
	$rules = \preg_split( "/\r\n|\n|\r/", carbon_get_theme_option( 'rtw_import_rules' ) );
	if ( ! $rules ) {
		return [];
	}

	$matching_pages = [];
	foreach ( array_splice( $data, 0 ) as $page ) {
		if ( isset( $page['children'] ) ) {
			foreach ( $page['children'] as $block ) {
				if ( in_array( $block['string'], $rules ) ) {
					ray( $block['string'] );
					$matching_pages[] = $page;
				}
			}
		}
	}

	return $matching_pages;
}

Rendering whole content tree

After we have a list of pages that match we need to render whole Roam content page that is in Markdown strings into HTML. Since I’ve been inspired by “Roam Blocks” plugin I’ve decided to reuse their function that does that: https://github.com/artpi/wp-roam-block/blob/main/endpoints.php#L82

First rendered page

With a bit more glue code I can already see how this plugin creates a new Note custom post types and inserts a page that I tagged with #publish:

Next steps

  • Figure out how to grab Title:: attribute if present so I don’t need to expose internal Roam page titles
  • Figure out what should be a bullet and what should be just a paragraph.
  • Publish to Github so maybe some other brave soul could give it a try

Islands vs. Streams as a learning model

How we learn in this digital age is broken, amazing, and weird at the same time. I see two major paradigms that are present in learning models: islands and streams.

Islands of learning are a traditional classroom approach. There’s a syllabus of reading, tasks to do, and lectures to listen to. It’s a very safe and guarded experience. Major online learning platforms such as Udemy or Coursera are bringing that model online. The other option is to learn through different streams of information. You read a chapter from a book, look at the YouTube video, do a short Skillshare course, and lurk on Instagram.

I’m trying to figure out how we can make these types of learnings more explicit to the learners. I’m also noticing that there is prestige attached to being a part of an island of learning. There’s just more status to say that you’re part of an expensive island instead of admitting that you’ve learned from many YouTube videos and blog posts.

There’s also a matter of getting good feedback loops in the process of learning. Let’s say you decide to learn about baking sourdough bread.

Islands way is to: 

  1. Read a book and try to follow instructions
  2. Go to a two-day workshop.
  3. Experiment with baking and talk to your family and a few close friends.
  4. Participate in a Facebook group for your workshop class

Streams way would be to:

  1. Find a 101 YouTube video from Tasty
  2. Read articles on the Perfect Loaf website
  3. Fail and discover “Beginner sourdough bakers” Facebook groups
  4. Post your pictures, get feedback from others, and with time get better
  5. Start posting on Instagram and get feedback from a global community of people

You only need to find the first few and you’ll organically find them by attracting people that are just a bit better than you. You’ll also start helping people that know a bit less than you.

6. Discover new sub-communities and repeat the process. Think of it as a journey and there’s no final destination.

The way I described streams is very community-driven. It requires a lot of vulnerability to consistently share failures instead of only your successes. Even with your best work you should approach it by asking how to improve it further.

I’m approaching learning from both perspectives and it’s frustrating in both cases. Islands make it hard to weave in resources from outside. Streams are often just these giants blobs of content that don’t make it easy to weave them into a coherent story and they’re hard to reference later.

Products that would help me on this learning journey

Figure out what’s the smallest and most streamable unit of content for each creative work. I’m thinking in terms of paragraphs and short video clips from longer videos. I’d like to have a way to easily assemble a learning trail of such content. Both for my reference and to be able to share it with others.

I’d like to relate these units of content to a larger community. Give a bit of context about the author and what’s the best audience for them.

Is there an inherent feedback loop that makes sharing such resources better with time? Can we develop assistive tools that will make it easier to suggest links to community-written FAQs or instructional videos?

Overall I still find that the process of collecting and curating learning is too high friction. There’s a lot of value in seeing the journey that one person took and we don’t make it easy for others to follow them. Our prevailing model is still mostly of top-down teaching and collaborative learning is still not a fundamental building block.

Matej Martinc explains Natural Language Processing

In Meaningful work interviews I talk to people about their area of work and expertise to better understand what they do and why it matters to them.

Matej Martinc is a Ph.D. researcher at “Jožef Stefan” Institute in the Department of Knowledge Technologies where he invents new approaches on how to work and analyze written text. He explained to me the basics of Natural Language Processing (NLP), why neural networks are amazing, and how one gets started with all of this. In the second half, he shared how he ended up in Computer Science with a Philosophy degree and why working for companies like Google is not something that interests him.

How do people introduce you?

They introduce me as a researcher at the IJS institute. I’m in the last year of my Ph.D. thesis research. I’m mostly working on Natural Language Processing (NLP). NLP is a big field and I’m currently exploring several different areas.

I initially started by automatically profiling text authors by their style of writing – we can detect their age, gender, and psychological properties. I also worked on automatic identification of text readability. We’ve also created a system to detect Alzheimer’s patients based on their writing.

Lately, I’ve been working on automatic keyword extraction and detecting political bias in word usage in media articles. I’m also contributing to research on semantic change – how word usage changes through time.

References to research that Matej is referencing throughout this interview. I encourage you to read them as they’re written in a very clear language.

Scalable and Interpretable Semantic Change Detection

[..] We propose a novel scalable method for word usage change detection that offers large gains in processing time and significant memory savings while offering the same interpretability and better performance than unscalable methods. We demonstrate the applicability of the proposed method by analyzing a large corpus of news articles about COVID-19

Zero-Shot Learning for Cross-Lingual News Sentiment Classification

In this paper, we address the task of zero-shot cross-lingual news sentiment classification. Given the annotated dataset of positive, neutral, and negative news in Slovene, the aim is to develop a news classification system that assigns the sentiment category not only to Slovene news, but to news in another language without any training data required. [..]

Automatic sentiment and viewpoint analysis of Slovenian news corpus on the topic of LGBTIQ+

We conduct automatic sentiment and viewpoint analysis of the newly created Slovenian news corpus containing articles related to the topic of LGBTIQ+ by employing the state-of the-art news sentiment classifier and a system for semantic change detection. The focus is on the differences in reporting between quality news media with long tradition and news media with financial and political connections to SDS, a Slovene right-wing political party. The results suggest that political affiliation of the media can affect the sentiment distribution of articles and the framing of specific LGBTIQ+ specific topics, such as same-sex marriage.

Can you start by explaining some background about NLP (Natural Language Processing) to start with?

As a first step, it’s good to consider how SVM (support vector machine) classifiers and decision tree techniques used for classification work. Very broadly speaking, they operate on a set of manually crafted features extracted from the dataset that you train your model on. Examples of that type of features would be: “number of words in a document” or a “bag of words model” where you put all the words into “a bag” and a classifier learns which words from this bag appear in different documents. If you have a dataset of documents, for which you know into which class they belong to (e.g., a class can be a gender of the author that wrote a specific document), you can train your model on this dataset and then use this model to classify new documents based on how similar these documents are to the ones in the dataset on which the model was trained. The limitation of this approach is that these statistical features do not really  take semantic relation between words into account, since they are based on simple frequency-based statistics.

About 10 years ago a different approach was invented using neural networks. What neural networks allow you to do is to work with unstructured datasets because you don’t need to define these features (i.e., classification rules) in advance. You train them by inputing sequences of words and the network learns on itself how often a given word appears closer to another word in a sequence. The information on each word is gathered  in a special layer of this neural network, called an embedding layer that is basically a vector representation that encodes how a specific word relates to other words. 

What’s interesting is that synonyms have a very similar vector representation. This allows you to extract relations between words. 

An example of that would be trying to answer: “Paris in relation to France” is the same as “Berlin in relation to (what?)”. To solve this question you can take the embedding of Paris, subtract the embedding of France and add embedding of Berlin and you’ll get an embedding as an answer – Germany. This was a big revolution in the field as it allows us to operationalize relations in the context of languages. The second revolution came when they invented transfer learning, a procedure employed for example in  the BERT neural network that was trained on BookCorpus with 800 million words and English Wikipedia with 2500 million words. 

In this procedure, the first thing you want to do is to train a language model. You want the model to predict the next word in a given sequence of words. You can also mask words in a given text and train the neural network to fill the gaps with the correct words. What implicitly happens in such training is that the neural network will learn about semantic relations between words. So if you’re doing this on a large corpus of texts (like billions of words in BERT) you get a model that you can use on a wide variety of general tasks. Because nobody had to label the data to do the training it means that it’s an unsupervised model.

Are you working with special pre-trained datasets?

I’m now mostly working with unsupervised methods similar to the BERT model. So what we do is to take that kind of model and do additional fine-tuning on a smaller training set  that makes it better suited for that specific research. This approach allowed us to do all of the research that I’m referencing here.

A different research area that doesn’t require additional training is to  employ clustering on the embeddings of these neural networks. You can take a corpus of text from the 1960s and another one from the 2000s. We can then compare how usage of specific embeddings (words) compare between these two collections of texts. That’s essentially how we can study how the semantic meaning of words changed in our culture.

Modern neural networks can also produce embedding for each usage of a word, meaning that words with more than one meaning have more than one embedding. This allows you to differentiate between Apple (software company) and apple (fruit). We used this approach when studying how different words connected to  COVID changed through time. We generated embeddings for each word appearance in the corpus of news about COVID and clustered these word occurrences into distinct word usages. Two interesting terms that we identified were diamond and strain. For strain, you can see the shift from using it in epidemiological terms (strain virus) to a more economic usage in later months (financial strain).

What we showed with our research is that you can detect changes even across short (monthly) time periods. There’s a limit to how accurately we can identify the difference. It’s often hard even for humans to decide how to label such data. We can usually get close to humane performance by using our unsupervised methods.

(both figures are from paper Scalable and Interpretable Semantic Change Detection)

Does this work for Non-English languages?

You can use the same technology with a non-English language and we’re successfully using it with Slovenian language. In the case of  viewpoint analysis of Slovenian news reporting, we’ve discovered a difference in how the word deep is used in  different context. Mostly because of the deep state that became a popular topic in certain publications.

For our LGBTIQ+ research, we can show that certain media avoids using the word marriage in the context of LGBTIQ+ reporting and replaces it with terms like  domestic partnership. They’re also not  discussing LGBTIQ+ relationship within the context of terms such as family. We can detect the political leaning of the media based on how they write about these topics.

We just started with this research on the Slovenian language so we expect that we’ll have much more to show later in the year.

(figure is from paper Automatic sentiment and viewpoint analysis of Slovenian news corpus on the topic of LGBTIQ+)

So far you’ve talked about analysis and understanding of texts. What other research are you doing?

We’re working on models for generating texts as part of the Embeddia project. The output of this research also works with the Slovenian language.

We’re also investigating if we can transfer embeddings between languages. We have a special version of the BERT neural network that has been trained on 100+ different language Wikipedias. What we’ve found out is that you can take a corpus of texts in the English language, train the model on  it to, for example, detect the gender of the author, and then use that same model to predict the gender of the author of some Slovenian text. This approach is called a zero-shot transfer.

How approachable is all this research and knowledge? Do I need a Ph.D. to be able to understand and use your research?

It takes students of our graduate school about a year to become productive in this field. The biggest initial hurdle is that you need to learn how to work with neural networks.

Good thing is that we now have very approachable libraries in this field. I’m a big fan of PyTorch as it’s well integrated with the Python ecosystem. There’s also TensorFlow that’s more popular in the industry and less in research. I found it harder to use for the type of work we’re doing and harder to debug. With PyTorch it takes about a month or two for our students to understand the basics.

In our context, it’s not just about using the existing neural networks and methods. Understanding the science part of our field and how to contribute via independent paper writing and publishing it’s usually about 2 years.

How easy is it to use your research in ‘real-world’ applications?

We have some international media companies that are using our research in the area of automatic keyword extraction from text. We’re helping them with additional tweaking of our models.

Overall we try to publish everything that we do under open access licenses with code and datasets publicly available.

What we don’t do is maintain our work in terms of production code. It’s beyond the scope of research and we don’t have funding to do it. It’s also very time-consuming and it doesn’t help us with our future research. That’s also what I like about scientific research. We get to invent things and we don’t need to maintain and integrate them. We can shift our focus to the next research question.

So in practice, all of our research is available to you but you’ll need to do the engineering work to integrate it with your product.

Let’s shift a bit to your story and how you got into this research. How did you get here?

I first graduated in philosophy and sociology in 2011, at the time when Slovenia was still recovering from the financial crisis. While I considered Ph.D. in philosophy I decided that there are not many jobs for philosophers. That’s why I’ve enrolled in a Computer Science degree that offered better job prospects.

During my Computer Science studies, I was also working in different IT startups. I quickly realized that you don’t have a lot of freedom in such an environment. Software engineering was too constrained for me in terms of what kind of work I could do.

After I graduated I took the opportunity to do Erasmus Exchange and I went to University in Spain. In that academic environment, I found the opposite approach. I received a dataset, a very loose description of a problem, and complete freedom to decide on how I’m going to approach and solve the problem.

When I returned to Slovenia I decided to apply to a few different laboratories inside IJS to see if I could continue with academic research. I’ve got a few offers and accepted the offer from the laboratory where I’m working today. 

I also decided to focus on NLP and language technologies as I’m still interested in doing philosophical and sociological research. Currently, I have the freedom to explore these topics in my research field without too many constraints. I’m also really enjoying all the conferences and travel that comes with it. Due to the fast-changing nature of my field, all the cutting-edge research is presented at conferences, and publishing in journals is just too slow. It takes over a year to publish a paper but there’s groundbreaking research almost monthly.

How do you see research done at FAANG (Facebook, Amazon, Apple, Netflix, Google) companies? We know that they’re investing a large amount of money into this field and have large research teams.

They’re doing a lot of good research. At the same time, they’re also often relying more on having access to a large number of hardware resources that we don’t. This can be both a blessing and a curse. At the moment I don’t see their research being that much better from the findings from universities. Universities are also more incentivized to develop new optimization techniques as they can’t use brute hardware force for their research.

Are you considering working for a FAANG company after your Ph.D.?

Not really. I already have a lot of freedom in my research and I can get funding to explore the areas that interest me. If I would work inside a FAANG company I would need to start at the bottom of the hierarchy and also be limited by their research agenda.

I also really like living in Slovenia and I don’t want to relocate to another country. At the same time, I’m excited about potential study/researchexchanges as I enjoy collaborating with researchers at foreign institutions.

What are some good resources to follow in your field?

You can follow the current state of the art at:

Papers describing paradigm shifts in the field of NLP:

Unsupervised language model pretraining and transfer learning: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

What I learned from talking with Matej

  • Recognizing what kind of work makes you happy allows you to optimize your job or clients so that you do such work.
  • Natural Language Processing is a very approachable technology and not something that only big companies can use.
  • There are many opportunities to bring research findings into the industry. It does require expertise and connections to both fields.
  • These technologies now also work for the Slovenian language.