Linear Algebra and Machine Learning for Facebook based optimization AI

Facebook: Linear Algebra and ML for AI based Algorithm

The main idea that we are going to be talking about here is how Facebook uses Linear Algebra and machine learning on it’s news feed AI algorithm and of course how it works on us. Let’s try to understand how this functions technically.  But, first let us start with something that almost all social media platforms use these days and it is known as a recommender system. 

Recommender system.

 We will start with a definition. A recommender system is a technology that is deployed in the environment where items (products, movies, events, articles) are to be recommended to users (customers, visitors, app users, readers) or the opposite. Typically, there are many items and many users present in the environment making the problem hard and expensive to solve. This is why we have the recommender system in place.

Now, before beginning with the types of recommender systems out here, we would like to consider here the possibility of hidden preferences that the users share. that users may share one or more such hidden preferences.

 1)  One such preference is hitting like on the same post, reading the same article or resharing a particular post. Here, the hidden preference that we are considering is where users are interacting with events.

2)  Another hidden preference that we can consider is that users are more likely to respond in the same way to similar events.

         

         

The types of Recommender Systems are as follows:

Knowledge based recommender systems

Both users and events have attributes. The more you know about your users and events, the better results can be expected. For example:

Item: TV230 {
  "name": "Television TV230",
  "short description": "HD LED TV",
  "long description": "  Enjoy a movie with your family on the weekend with this HD television. With an 81cm (32) display, you can view every single detail with rich detail and clarity. This LED TV produces a resolution of 1366 x 768 pixels with a refresh rate of 60Hz to display crisper images and fluid picture movement”

"price": 150,
  "categories": ["Electronics", "Televisions"]}

Such attributes are very useful and data mining methods can be used to extract knowledge in forms of patterns that are used for recommendation.

Content based recommender systems

Such systems are recommending items similar to those a given user has liked in the past, regardless of the preferences of other users. Basically, there are two different types of feedback.

Explicit feedback is provided by users in form of clicking the “like”/”dislike” buttons etc. In many cases, it is hard to obtain explicit feedback data, simply because the users are not willing to provide it. Instead of clicking “dislike” for an event which the user does not consider interesting, he/she will rather leave the web page .

Implicit feedback data, such as “user viewed an event”, “user finished reading the article” or “user ordered a product”, however, are often much easier to collect and can also help us to compute good recommendations. Types of implicit feedback may include:

Interactions(implicit feedback)

1) User has viewed an event.

2) User has read an article

3) User has viewed the event's detail

And we can expect better performance if the feedback is good.

Content based recommenders work with the past interactions of a given user and do not take 

other users into consideration.

 MACHINE LEARNING AND LINEAR ALGEBRA FOR FACEBOOK BASED

 RECOMMENDER  SYSTEMS.

 So, to understand how ML and linear algebra go hand in hand we need to understand some basic

 terminlogies.

 The first method that we are going to be talking about here is called collaborative filtering.

Thus, here we are relying only on observed user behavior to make recommendations. 

Let us try to understand how collaborative filtering actually works: (image: fb.code)

      

     MATRIX FACTORIZATION BASED RECOMMENDER SYSTEMS                                            

We need to calculate the implicit or explicit rating of the user to provide a better performing recommender system . So, every entry in our matrix captures a user's reaction to a given event. If a user has never rated an event, the matrix entry is zero. Often, in these type of matrices, the majority of the entries are zero. We're assuming that there's a set of features common to all of these events and events differ in how they express these features. Each user has their own reaction to each feature, independent of the events.. These features are hidden factors. Now, these features will transform the matrix to represent the hidden factors. This process is called low-rank approximation. It's the process of compressing the Sparsity of a matrix (sparse information) into a much lower dimensional space. Then, we can calculate the rating of a given user for a given event by taking the dot product of two vectors which can be done by means of a neural network. The expectation is that unknown user-to-event ratings can be approximated by dot products of corresponding feature vectors, as well. The simplest form of objective function, which we want to minimize, is:(image: fb.code.com)

Here, r are known user-to-event ratings, and x and y are the user and event feature vectors that we are trying to find. As there are many free parameters, we need the regularization part to prevent overfitting and numerical problems, with gamma being the regularization factor.

Scalable CF by Facebook

Facebook’s own recommender system uses Matrix Factorization and can handle massive datasets with a 100 Billion ratings. It can be extended to other approaches as well.

Dot products by means of neural networks.

Now, with the output of the dot product of the vectors, we can represent it by the means of a neural network. We can use a weight matrix to optimize these hidden preferences and their predictions over time. We can also introduce a loss function that will determine the accuracy of our prediction. Lastly we can use gradient descent which would update our weights and minimize our loss(loss function can be implemented in many ways- least square method or MSE) and improve the prediction which in turn would update the user’s rating resulting in a highly improved recommender system. This canbe

used as one of the methods and many more methods are applicable. 

CONCLUSION

Facebook makes money by running ads. Essentially it’s not only you watching the advertisement but the advertisement is also watching you. Facebook’s AI optimized ad algorithm functions just like that. It knows who are in a relationship, got married recently, who likes a specific kind of chocolate, who are going toward depression  etc. Another example: Making people argue results in more engagement and more attention and this means more ad revenue. This occurs because of the mass surveillance Facebook does on its users. The AI optimized algorithm does just that. It controls our behaviour by creating a newsfeed more suited to individual interests.  So, with that being said we as humans need to move more cautiously these days  and be more aware of AI based algorithms so that it does not manipulate our thinking. The more aware we are, the more protected we would be.  

More reading material and some of my sources.

https://code.fb.com/core-data/recommending-items-to-more-than-a-billion-people/

https://medium.com/recombee-blog/recommender-systems-explained-d98e8221f468

http://humanetech.com/problem#the-way-forward

https://bitlab.cas.msu.edu/papers/news_feed_fyi_chi17_lbw.pdf

https://github.com/llSourcell/Stream-Framework.