def get_all_corners(bottom_left_corner, top_right_corner): n_dim = len(bottom_left_corner) mins = np.minimum(bottom_left_corner, top_right_corner) maxs = np.maximum(bottom_left_corner, top_right_corner) intervals = np.stack([mins, maxs], -1) indexing_bits = np.array(list(itertools.product([0, 1], repeat=n_dim))) corners = np.stack([intervals[x][indexing_bits.T[x]] for x in range(n_dim)], -1) return corners </code></pre> – Ilja

Visualizing years of messaging on Facebook

2021-04-26T00:00:00+00:00

Back in winter 2016, I have discovered /r/DataIsBeautiful</a> (and /r/DataIsUgly</a>) and when the holidays came, I’ve set the goal of learning D3.js and creating a nice visualization myself. I saw a reddit post</a> and immediately thought that was something I wanted to try to replicate. It took me few full days of learning, but I was really happy with the result</a>.

As of now, more than 4 years have passed and it would be interesting to see how my messaging patterns and habits changed. Unfortunately, back then I was not as careful with my side projects as I am now, and I lost all the code. Moreover, I (happily) haven’t touched Javascript for a few years now, and I am hoping to continue with that. For my 2021 spring vacation, I’ve set a goal of redoing the visualization in Python, with new data and document everything properly for a blog post, and hopefully finding something new along the way. If you are reading this on my website, the experiment is a success!

Downloading Facebook data</h2>
First, we need to obtain the messaging data. Few years ago I had to use some third-party scripts that would scroll through my messages and save them one by one. The technology (or more precisely, enforcement of GDPR) moves forward and now Facebook provides a nifty tool with which you can download the data directly. Go to facebook.com/dyi</a>, select Messages, JSON format and press Create File. Depending on the amount of messages you have, it will take anywhere between half an hour and a day to receive the download link.
While not needed for this project and would probably take longer to create data archive, I highly recommend to request all of the possible data, not just messages. It is quite eye opening to see how much exactly information Facebook collects, and not just on their websites, but from third parties</a>, even when you are not logged in</a>.

Parsing and filtering messages</h2>

Messages are stored in message_*.json</code> files in messages</code> folder of your Facebook data archive. Let’s read all of them (and fix facebook’s mojibake</a>) and make a list of all messages that we can find:

import </span>json
</span>from </span>pathlib </span>import </span>Path
</span>
</span># fix mojibake: https://stackoverflow.com/a/62160255/1668421
</span>def </span>parse_obj</span>(</span>obj</span>):
</span>    </span>if </span>isinstance</span>(obj, str):
</span>        </span>return </span>obj.</span>encode</span>('</span>latin_1</span>').</span>decode</span>('</span>utf-8</span>')
</span>
</span>    </span>if </span>isinstance</span>(obj, list):
</span>        </span>return </span>[</span>parse_obj</span>(o) </span>for </span>o </span>in </span>obj]
</span>
</span>    </span>if </span>isinstance</span>(obj, dict):
</span>        </span>return </span>{key: </span>parse_obj</span>(item) </span>for </span>key, item </span>in </span>obj.</span>items</span>()}
</span>
</span>    </span>return </span>obj
</span>
</span>message_files = [f </span>for </span>f </span>in </span>Path</span>('</span>path/to/messages/folder</span>').</span>rglob</span>('</span>message_*.json</span>')]
</span>messages = []
</span>for </span>m </span>in </span>message_files:
</span>    </span>with </span>open</span>(m, '</span>r</span>') </span>as </span>f:
</span>        messages += </span>parse_obj</span>(json.</span>load</span>(f)['</span>messages</span>'])
</span></code></pre>
Each element of messages</code> is a dictionary and depending on the type, fields can differ. In my archive I have found five types:</p>
>>> </span>set</span>([m['</span>type</span>'] </span>for </span>m </span>in </span>messages])
</span>['</span>Share</span>', '</span>Subscribe</span>', '</span>Unsubscribe</span>', '</span>Generic</span>', '</span>Call</span>']
</span></code></pre>
Of these five, I am interested only in Share</code> (sending links) and Generic</code> (chat messages). Subscribe</code> and Unsubscribe</code> seem to be used only for Facebook bots, while Calls</code> are self-explanatory.</p>
messages = </span>list</span>(</span>filter</span>(</span>lambda </span>d</span>: d['</span>type</span>'] in ['</span>Share</span>', '</span>Generic</span>'], messages))
</span></code></pre>
Let’s inspect those messages and see how many unique fields I have in my data and whether they are always populated. I have to note that this is what I have in my data. It is probably not the full list of all possible fields and YMMV.</p>
fields = </span>list</span>({field </span>for </span>message </span>in </span>messages </span>for </span>field </span>in </span>message})
</span>
</span># There is definitely a one-line list comprehension that can replace this snippet
</span>counter = dict.</span>fromkeys</span>(</span>list</span>({field </span>for </span>message </span>in </span>messages </span>for </span>field </span>in </span>message}), </span>0</span>)
</span>for </span>f </span>in </span>counter:
</span>    </span>for </span>m </span>in </span>messages:
</span>        </span>if </span>f in m:
</span>            counter[f] += </span>1
</span>
</span>>>> {f: counter[f] == </span>len</span>(messages) </span>for </span>f </span>in </span>counter}
</span>{
</span>    '</span>files</span>': </span>False</span>, 
</span>    '</span>content</span>': </span>False</span>, 
</span>    '</span>timestamp_ms</span>': </span>True</span>, 
</span>    '</span>videos</span>': </span>False</span>, 
</span>    '</span>gifs</span>': </span>False</span>, 
</span>    '</span>photos</span>': </span>False</span>, 
</span>    '</span>reactions</span>': </span>False</span>, 
</span>    '</span>audio_files</span>': </span>False</span>, 
</span>    '</span>type</span>': </span>True</span>, 
</span>    '</span>is_unsent</span>': </span>True</span>, 
</span>    '</span>ip</span>': </span>False</span>, 
</span>    '</span>sticker</span>': </span>False</span>, 
</span>    '</span>sender_name</span>': </span>True</span>, 
</span>    '</span>share</span>': </span>False
</span>}
</span></code></pre>
It seems that fields timestamp_ms</code>, is_unsent</code>, sender_name</code> and type</code>  are the only ones that are populated all messages.</p>
Now, let’s filter by time range, for example let’s filter out messages that are are not from 2020. Note, Facebook stores timestamps in milliseconds.</p>
import </span>time
</span>
</span>time_from = time.</span>mktime</span>(time.</span>strptime</span>('</span>01/01/2020</span>', '</span>%d</span>/</span>%m</span>/</span>%Y</span>')) * </span>1000
</span>time_to = time.</span>mktime</span>(time.</span>strptime</span>('</span>01/01/2021</span>', '</span>%d</span>/</span>%m</span>/</span>%Y</span>')) * </span>1000
</span>
</span>messages = </span>list</span>(</span>filter</span>(</span>lambda </span>d</span>: (d['</span>timestamp_ms</span>'] > time_from) and (d['</span>timestamp_ms</span>'] < time_to), messages))
</span></code></pre>
Messaging heatmap</h2>
Now that we have only the messages we need for the visualization, let’s recreate the graph that I mentioned before with Pycairo</a>, Python bindings for cairo graphics library. The API is pretty similar to D3.js and quite pythonic (I think I will use Pycairo again in the future!). The toughest part was basic trigonometry for finding correct coordinates on the circle and tweaking sizes and offsets to make it look better.</p>
In short, the algorithm for drawing is following:</p>

Draw background circle</li>
Draw inner circles, month delimiters and month names</li>
Draw inner text</li>
For each message:

Convert message’s timestamp to the day of the year and a fraction of the day</li>
Draw a small semi-transparent stroke on the circle in the appropriate day of the year and time of the day</li>
</ol>
</li>
</ol>




	

</a>
2020 was a very unusual year for all of us. I was lucky enough to (happily) get stuck in Japan with my girlfriend, essentially maxing out tourist visa and staying there for almost 6 months. This is clearly visible on the heatmap: I left for Japan mid-March and returned back home mid-August.</p>
Analyzing messages data</h2>
While my initial goal was to recreate the heatmap, I have decided to try to find some other information in the messages data.</p>
Unique people you have chatted with:</p>
len</span>(</span>set</span>([m['</span>sender_name</span>'] </span>for </span>m </span>in </span>messages]))
</span></code></pre>
How many unsent messages you have in your chats:</p>
len</span>([m </span>for </span>m </span>in </span>messages </span>if </span>m['</span>is_unsent</span>']])
</span></code></pre>
How many reactions you and your friends have put:</p>
len</span>([m </span>for </span>m </span>in </span>messages </span>if </span>'</span>reactions</span>' in m])
</span></code></pre>
Message content</h3>
Since the content of the message is also included, you can even further and analyze the messages themselves. For example, we can count occurences of every word:</p>
from </span>collections </span>import </span>Counter
</span>import </span>string
</span>
</span>c = </span>Counter</span>()
</span>for </span>m </span>in </span>messages:
</span>    </span>if </span>'</span>content</span>' in m:
</span>
</span>        </span># remove punctuation
</span>        content = m['</span>content</span>'].</span>translate</span>(str.</span>maketrans</span>('', '', string.punctuation)) 
</span>
</span>        </span># count occurences in lowercase
</span>        </span>for </span>w </span>in </span>content.</span>split</span>():
</span>            c[w.</span>lower</span>()] += </span>1  
</span></code></pre>
Later, we can make a bar plot of 25 most common words:</p>
import </span>matplotlib.pyplot </span>as </span>plt 
</span>mc = c.</span>most_common</span>(</span>25</span>)
</span>plt.</span>bar</span>([x[</span>0</span>] </span>for </span>x </span>in </span>mc], [x[</span>1</span>] </span>for </span>x </span>in </span>mc])
</span>plt.</span>show</span>()
</span></code></pre>
It doesn’t really follow Zipf’s law</a>, which states that frequency of any word is inversely proportional to its rank, but it is reminiscent of a log graph and heavy-tailed as expected.</p>


	
Location data</h3>
Another interesting discovery is that Facebook seem to save IP addresses from where I logged in to Messenger. According to DYI their info page</a>, they call it “Your recent message activity from specific IP addresses”.</p>
How many unique IPs facebook has seen you message from:</p>
set</span>([m['</span>ip</span>'] </span>for </span>m </span>in </span>messages </span>if </span>'</span>ip</span>' in m])
</span></code></pre>
Going further, you can find approximate location of each of the IP address with tools like ipinfo.io</a>, which recently launched bulk upload tool. Plotting those locations on the world map, you get something like this:</p>


	
– Ilja</p>

Jeff Tupper’s quine

2021-04-05T00:00:00+00:00

Yesterday, a friend sent me Numberphile’s video, “The ‘Everything’ Formula”</a>

</iframe> </div>

The video describes a neat formula that acts as a plotter for any arbitrary bitmap of size 106x17. For some reason, it is often referred to as “Tupper’s self-referential formula”:

$$\frac{1}{2} < \left\lfloor \mathrm{mod}\left(\left\lfloor \frac{y}{17} \right\rfloor 2^{-17 \lfloor x \rfloor - \mathrm{mod}\left(\lfloor y\rfloor, 17\right)},2\right)\right\rfloor$$

Jeff Tupper’s original paper</a> indeed contains this formula (Figure 13), but it is listed only as an example of two-dimensional graphing and it is never described as self-referential. While it is a cool mind-scratcher, it is definitely not self-referential, but a quine</a>. It can produce itself in a different modality, but only when looked at the right place on the infinite x-y plane. I like Arvind Narayanan’s comparison</a> with the infinite monkey theorem</a>. Can a monkey hitting keys at random for an infinite amount of time eventually type Shakespeare’s Hamlet? Yes, almost surely. The question is where can we find it in the monkey’s infinite string. Can a function that describes every possible 106x17 (1802!) bitmap somewhere on x-y plane contain some representation of the original function? Sure can, but we need to know where exactly.

There are dozens of articles and blog posts that go in details of how does it work</a> and some even propose an actual self-referential formula</a>. The posts I have linked explain the theory behind the formula, and much better than I can. Instead, on this quiet Easter Monday I want to demonstrate how I have implemented both finding where can we find the bitmap we are looking for, as well as plotting this bitmap, all in modern Python.

Plotting the formula</h2>

Let’s start with the plotting. The image that we are looking for should be at 0 < x < 106</code> and k < y < k + 17</code>. In the original paper, k</code> is defined as

960 939 379 918 958 884 971 672 962 127 852 754 715 004 339 660 129 306 651 505 519 271 702 802 395 266 424 689 642 842 174 350 718 121 267 153 782 770 623 355 993 237 280 874 144 307 891 325 963 941 337 723 487 857 735 749 823 926 629 715 517 173 716 995 165 232 890 538 221 612 403 238 855 866 184 013 235 585 136 048 828 693 337 902 491 454 229 288 667 081 096 184 496 091 705 183 454 067 827 731 551 705 405 381 627 380 967 602 565 625 016 981 482 083 418 783 163 849 115 590 225 610 003 652 351 370 343 874 461 848 378 737 238 198 224 849 863 465 033 159 410 054 974 700 593 138 339 226 497 249 461 751 545 728 366 702 369 745 461 014 655 997 933 798 537 483 143 786 841 806 593 422 227 898 388 722 980 000 748 404 719 </blockquote> A straight forward way to plot this is to evaluate the function for each x,y point at this separately. Instead, we can vectorize computations with NumPy and use Pillow for loading and saving images:import numpy as np from PIL import Image K = ... # omitted for readability # build a meshgrid for X coordinates from 0 to 106 and Y coordinates from K to K+17 x, y = np.meshgrid(range(106), range(K, K+17)) # evaluate Tupper's formula for all points at once result = 0.5 < ((y // 17) // (2 ** (17 * x + y % 17))) % 2 # in my implementation we need to mirror the image due to coordinates mismatch result = np.fliplr(result) # convert into an image and save to disk Image.fromarray(result).save('original.png') </code></pre> Here’s the “self-replicating” result: We have confirmed that the approach works, now let’s see how we can find more interesting bitmaps! Finding k for arbitrary images</h2> The algorithm is quite simple: Convert the 106x17 image of interest into a thresholded binary image where each pixel has value 0 or 1.</li> Flatten the image column-wise into a single binary number that consists of 106*17=1802 ones or zeros.</li> Convert this number into base-10 and multiply by 17.</li> </ol> Let’s translate this into code. Again, we’ll use NumPy and PIL. import numpy as np from PIL import Image # load the image and convert it to a numpy array image = np.array(Image.open(path_to_image)) # threshold the image: convert the array to binary and then back to int image = image.astype(bool).astype(int) # transpose (image must be encoded column-wise), mirror (to match original coordinates frame) and flatten image = np.fliplr(image.T).flatten() # represent array as one integer in base 10 and multiply it by 17 K = int(''.join(map(str, image)), 2) * 17 # for readability let's split the found number by 3 characters from textwrap import wrap print(f'Found K: {" ".join(wrap(str(K), 3))}') </code></pre> Let’s see if this will work for the OG: Found K: 960 939 379 918 958 884 971 672 962 127 852 754 715 004 339 660 129 306 651 505 519 271 702 802 395 266 424 689 642 842 174 350 718 121 267 153 782 770 623 355 993 237 280 874 144 307 891 325 963 941 337 723 487 857 735 749 823 926 629 715 517 173 716 995 165 232 890 538 221 612 403 238 855 866 184 013 235 585 136 048 828 693 337 902 491 454 229 288 667 081 096 184 496 091 705 183 454 067 827 731 551 705 405 381 627 380 967 602 565 625 016 981 482 083 418 783 163 849 115 590 225 610 003 652 351 370 343 874 461 848 378 737 238 198 224 849 863 465 033 159 410 054 974 700 593 138 339 226 497 249 461 751 545 728 366 702 369 745 461 014 655 997 933 798 537 483 143 786 841 806 593 422 227 898 388 722 980 000 748 404 719 </blockquote> That’s exactly the same string as above, yay! Time to find find some interesting k’s. Other k</h2> As a tribute to the author, let’s find k of Jeff Tupper’s name: Found K: 385 815 185 034 138 158 985 870 922 709 187 771 741 984 863 903 001 560 903 755 944 700 527 424 584 051 315 029 117 989 906 439 493 180 632 249 939 860 962 369 068 130 333 226 454 308 011 056 898 770 189 860 026 183 100 209 457 279 325 986 175 729 482 270 868 521 087 312 425 053 904 627 001 175 969 513 613 540 074 140 325 118 540 773 528 302 948 665 290 791 255 891 024 244 612 978 717 349 953 081 144 003 722 839 141 078 743 527 892 814 519 951 422 463 077 169 240 686 548 347 390 436 099 422 493 685 020 088 219 659 235 193 486 350 953 092 824 772 778 233 932 879 750 089 176 444 321 163 669 351 665 248 816 840 127 309 058 081 171 960 635 932 943 715 857 693 920 045 698 334 277 341 544 448 </blockquote> Mustering all my artistic skill, I managed to draw something easily recognizable even on such a small image: Found K: 126 142 104 213 220 484 299 803 993 442 062 205 409 077 505 220 633 202 015 858 972 307 182 501 421 645 782 823 217 849 945 196 989 219 566 869 642 586 856 002 258 241 479 301 667 237 018 424 945 404 030 790 228 025 510 765 380 030 265 144 252 981 546 142 589 226 719 086 965 652 739 520 845 129 864 851 583 864 577 815 519 135 290 434 195 548 911 229 646 938 854 682 829 338 643 067 177 226 847 816 952 758 734 166 379 981 459 848 756 683 277 095 647 000 957 329 360 499 307 384 746 219 787 041 087 624 524 570 154 355 080 970 812 831 032 393 680 420 031 478 481 534 582 784 </blockquote> We can go futher and add support for RGB(A) images by grayscaling and thresholding before converting image to a number, allowing us to find k</code> of some fun things. if image.ndim == 3: # thresholding value was chosen experimentally ¯\_(ツ)_/¯ image = image[..., :3].mean(-1) > (image.mean() / 1.5) </code></pre> For example, we can try to find k of a cat who is being yelled at by a woman</a>: Found K: 851 030 412 628 336 556 165 809 369 962 109 912 804 161 242 368 430 933 212 288 014 494 536 807 542 740 734 381 671 364 065 155 879 388 013 869 494 956 962 061 134 763 310 263 193 385 575 072 149 137 089 604 705 201 819 322 847 554 746 397 401 554 468 442 840 234 615 665 452 154 695 967 266 451 351 629 936 505 011 951 742 325 309 194 431 147 721 355 712 298 208 733 534 626 570 288 417 823 332 710 025 407 637 465 753 296 616 837 452 115 490 622 693 556 930 112 201 054 318 590 446 660 494 224 496 302 101 506 150 764 129 717 619 346 698 940 226 775 371 940 997 279 460 630 181 000 578 484 533 391 638 594 731 484 579 785 972 285 338 008 247 656 925 936 746 496 Result: </blockquote> Or a surprised pikachu: Found K: 100 471 937 400 773 042 105 488 795 487 079 558 956 078 370 881 471 858 986 932 192 035 983 663 503 559 690 104 085 241 627 816 344 060 029 535 358 169 730 565 614 348 280 274 258 391 956 093 706 487 773 600 207 321 552 633 572 512 727 519 083 578 975 536 089 240 998 241 402 260 568 596 955 034 368 758 047 736 965 380 237 422 807 289 192 867 659 494 212 761 961 844 085 991 026 277 158 315 733 803 429 051 940 595 481 874 380 651 187 292 245 665 209 511 827 991 598 028 036 167 858 029 884 225 988 105 814 239 390 403 337 695 473 230 329 464 921 830 026 338 051 548 823 510 917 702 632 449 767 686 964 811 819 836 113 753 470 019 547 288 874 741 950 634 085 738 923 194 014 636 840 792 896 8 Result: </blockquote> All in all, that was a fun one day project! – Ilja

gubins.lv

Finding hyperrectangle vertex coordinates

Visualizing years of messaging on Facebook

Jeff Tupper’s quine

Developing a custom thumbnailer for Nautilus

gubins.lv

Finding hyperrectangle vertex coordinates

Visualizing years of messaging on Facebook

Analyzing messages data</h2> While my initial goal was to recreate the heatmap, I have decided to try to find some other information in the messages data.</p> Unique people you have chatted with:</p>

Message content</h3> Since the content of the message is also included, you can even further and analyze the messages themselves. For example, we can count occurences of every word:</p>

Jeff Tupper’s quine

Finding k for arbitrary images</h2> The algorithm is quite simple:</p>

Developing a custom thumbnailer for Nautilus