lordievader's blog

Soft Reboot: keep your containers running while your image-based Linux host gets updated

Sat, 03 Feb 2024 14:26:09 +0000

Soft Reboot

Talk by Luca Boccassi.

This talk is about a new systemd feature. Soft reboot. Targeted towards performance, service interruption. Service downtime should be minimized.

This should pair nicely with kernel ’live patching'.

For the kernel ’nothing at all is happening’. However, for userspace it is a full shutdown/reboot.

With a File Descriptor Store applications can buffer connections to keep them alive while the system goes for a soft-reboot.

The soft-reboot allows containers to keep running. However, they need to be disconnected from the rootfs.

Demo shows an executable (‘sleep infinity’) in a Podman container. The system goes for a soft-reboot, restarts all the services. And afterwards the executable is shown to still be running, with the same pid it had before.

Conclusion

Interesting feature. Judging from the demo it seems to work well. What I mainly wonder after this talk, is still the same thing I had before this talk, how often do you want to reboot your system this way. Often the reason for me to reboot is updating the kernel. Services and applications can be restarted on their own.

References

Soft Reboot: keep your containers running while your image-based Linux host gets updated

Using chroots in a single Linux Container as an alternative to docker-compose

Sat, 03 Feb 2024 13:35:29 +0000

Using chroots in a single Linux Container as an alternative to docker-compose

Talk by Aiden McClelland (details linked in the Reference section below).

The talk is centered around StartOS. An operating system designed to enable non-technical users to self-host open source software.

The OS hosts this via containers. With the base philosophy ‘one container = one service/app’. This makes managing IP addresses easier, and managing resources easier.

The OS uses LXC to manage the containers. Aiden claims this is more composable than docker compose or podman.

There is a root image which all containers share. This image is a nodejs application which runs the package manager. And ultimately runs the application in a chroot.

The service api (inside the nodejs) controls the service. Various init commands, but also package management commands.

The OS mounts the same /dev, /proc and /sys inside all containers. I’m wondering if this is a good idea.

Using a whole bunch of Javascript they managed to expose multiple images inside a single container. While it is easier to make one unified service (which requires multiple components) I’m wondering if this is the way to go. Setting up a docker-compose project with multiple services to offer a uniting application is not too difficult.

Conclusion

Interesting talk to expose different images inside one and the same container. However, I question what the real benefits of this are. Or put otherwise, what downsides of running multiple containers are such that that makes this worthwhile.

I also seriously question the use of Javascript as the managing code.

References

Using chroots in a single Linux Container as an alternative to docker-compose

Where have the women of tech history gone

Sat, 03 Feb 2024 10:00:16 +0000

Where have the woman of tech history gone

First talk of this year’s FOSDEM (not counting the opening talk).

Talk by Laura Durieux (see the link at the References at the bottom of the page for more details).

The talk started with the question why women were underrepresented in the syllabus. This lead to the question why there is a focus on men, and who decides what makes it to the syllabus.

‘Computer’ was a job. A mathematical profession.

Nicole-Reine Lepaute, one of the first mentioned women in the literature. Calculated the return of the Haley’s comet.

In the 19th century women were forced to resign when they got married.

Ada Lovelace, was famous for her notes on the Analytic Engine. She wrote the first algorithm. Laura considers her the mother of computer science.

Annie Jump Cannon, developed a method for classifying starts, which is still used today. She classified 5.000 stars per month.

Grete Herman pioneered in quantum mechanics.

Between the ’40s and ’70s women were widely hired as coders. No certification required, and a shortage of manpower.

Eniac girls, the operators of computers. Early ancestors of debuggers. The hardware was a man’s job. However, as operators women needed to have an in-depth knowledge of the hardware. These girls had a mathemetics degree in common. The machines were really a bunch of cables. And these women at times needed to figure out where a bug (in the mess of cables) would be.

Hedy Lamarr, invented FHSS. Technology still used in Wi-FI and Bluetooth. A renowned movie star. Quite an unusual career change.

In the ’50s the stereo types of male programmers emerged.

Grace Hopper, known as the queen of software, created the compiler in 1949. One of the main creators of the COBOL language.

After the ‘60 there was a decline in the number of women majoring in computer science.

In the ’80s with the introduction of the personal computer the idea that to be a programmer you needed countless hours in front of a computer screen emerged. Pc’s were typically put in the son’s room with the father as the coach. Really focussed on the males rather than the women.

Karen Spark Jones developed the TF-IDF method. Used by most search engines. As the method shows which keywords are important from a corpus. She also had the quote that ‘computing is much too important to be left to men’.

Conclusion

Laura gave a nice overview of the women in computer science history. Everyone in computer science should see such a talk as women have played a major role in the history of computing. It may make the IT field more acceptable to women. And on the other side, if women in general learn more about this kind of history more women may consider a career in the IT field.

References

Where have the women of tech history gone?

Zettelkasten notes

Mon, 29 Jan 2024 13:44:35 +0100

Zettelkasten notes

Somewhere during my Ph.D. I started to use the Zettelkasten style of note taking. For my research this was quite useful. Now in my programmer life after the Ph.D. it is less usefull as I can clearly see that it geared towards academics.

What is a Zettelkasten?

I’ve written a note on what a Zettelkasten is here.

Public notes

I’ve setup a link between my Zettelkasten repository and this website. When I tag a note ‘public’ it automatically becomes a post on this website. The reason I opted for a tag filter is that I don’t want to publish anything I take a note of. Sometimes these are private, sometimes these are just ramblings. But sometimes, it may be worth something to someone. The major downside of this approach is that there may be broken links. If a note refers to another note which is not public the link wil be served a 404 Not Found.

There is a distinction between regular posts, like this one, and notes coming from the Zettelkasten. The latter all have the tag ‘#public’. To get an overview of these notes you can view this page.

References

QJsonObject to QString

Thu, 22 Jun 2023 11:50:00 +0200

Created: 2023-06-22 11:50 #cpp #qt #json

With the following snippets you can convert a QJsonObject into a QString.

First convert the QJsonObject to a QJsonDocument. Then output the document as a QByteArray (the toJson method) and feed that to the QString init.

QJsonDocument doc(QJsonObject json);
QString jsonString(doc.toJson());

References

Proactive Threat Detection: A DNS based approach

Mon, 26 Sep 2022 00:00:00 +0200

The second publication for the TIDE project. It has received the Best Paper Award at NOMS 2018.

Snowshoe spam is a type of spam which is notoriously hard to detect. Differently from regular spam, snowshoe spammers distribute the volume among many hosts, in order to make detection harder. To be successful, however spammers need to appear as legitimate as possible, for example, by adopting email best practice like Sender Policy Framework (SPF). This requires spammers to register and configure legitimate DNS domains. Previous studies uses DNS data to detect spam. However, this often happens based on passive DNS data. In this paper we take a different approach. We make use of active DNS measurements, covering more than 60% of the namespace, in combination with machine learning to identify malicious domains crafted for snowshoe spam. Our results show that we are able to detect snowshoe spam domains with a precision of more than 93%. Also, we are able to detect a subset of the malicious domain 2?104 days earlier than the spam reputation systems (blacklists) currently in use, which suggest our method can give us a time advantage in the fight against spam. In a real-life scenario, we have shown that our results allow spam filter operators to block spam that would otherwise bypass their mail filter. A Realtime Blackhole List (RBL) based on our approach is currently deployed in the operational network of a major Dutch ISP.

Slides of the presentation are available here: pdf

Title	Melting the Snow: Using Active DNS Measurements to Detect Snowshoe Spam Domains
Authors	Olivier van der Toorn, Roland van Rijswijk-Deij, Bart Geesink, Anna Sperotto
Publication date	2018/4/23
Conference	NOMS 2018

Slides

Thu, 21 Oct 2021 14:52:57 +0200

Slides of the following conferences are available:

ANYway: Measuring the Amplification DDoS Potential of Domains (preprint)

Fri, 17 Sep 2021 01:00:00 +0100

DDoS attacks threaten Internet security and stability, with attacks reaching the Tbps range. A popular approach involves DNS-based reflection and amplification, a type of attack in which a domain name, known to return a large answer, is queried using spoofed requests. Do the chosen names offer the largest amplification, however, or have we yet to see the full amplification potential? And while operational countermeasures are proposed, chiefly limiting responses to ‘ANY’ queries, up to what point will these countermeasures be effective? In this paper we make three main contributions. First, we propose and validate a scalable method to estimate the amplification potential of a domain name, based on the expected ANY response size. Second, we create estimates for hundreds of millions of domain names and rank them by their amplification potential. By comparing the overall ranking to the set of domains observed in actual attacks in honeypot data, we show whether attackers are using the most-potent domains for their attacks, or if we may expect larger attacks in the future. Finally, we evaluate the effectiveness of blocking ANY queries, as proposed by the IETF, to limit DNS-based DDoS attacks, by estimating the decrease in attack volume when switching from ANY to other query types. Our results show that by blocking ANY, the response size of domains observed in attacks can be reduced by 57%, and the size of most-potent domains decreases by 69%. However, we also show that dropping ANY is not an absolute solution to DNS-based DDoS, as a small but potent portion of domains remain leading to an expected response size of over 2,048 bytes to queries other than ANY.

Title	ANYway: Measuring the Amplification DDoS Potential of Domains
Authors	Olivier van der Toorn, Johannes Krupp, Mattijs Jonker, Roland van Rijswijk-Deij, Christian Rossow, and Anna Sperotto
Publication date	October 2021
Journal	17th International Conference on Network and Service Management (CNSM 2021)

TXTing 101: Finding Security Issues in the Long Tail of DNS TXT Records

Tue, 01 Sep 2020 01:00:00 +0100

The DNS TXT resource record is the one that without doubt provide users with the most flexibility of content, as it is a largely unstructured. Although it might be the ideal basis for storing any form of text-based information, it also poses a security threat, as TXT records can also be used for malicious and unintended practices. Yet, we reckon that TXT records are often overlooked in security research. In this paper, we present the first structured study of the uses of TXT records, with a specific focus on security implications. We are able to classify over 99.54% of all TXT records in our dataset, finding security issues including accidentally published private keys and exploit delivery attempts. We also report our lessons learned while dealing with a large-scale, systematic analysis of TXT records.

Title	TXTing 101: Finding Security Issues in the Long Tail of DNS TXT Records
Authors	Olivier van der Toorn, Roland van Rijswijk-Deij, Tobias Fiebig, Martina Lindorfer, and Anna Sperotto
Publication date	September 2020
Journal	5th International Workshop on Traffic Measurements for Cybersecurity (WTMC 2020)

A Case of Identity: Detection of Suspicious IDN Homograph Domains Using Active DNS Measurements

Tue, 01 Sep 2020 00:00:00 +0100

The possibility to include Unicode characters in domain names allows users to deal with domains in their regional languages. This is done by introducing Internationalized Domain Names (IDN). However, the visual similarity between different Unicode characters - called homoglyphs - is a potential security threat, as visually similar domain names are often used in phishing attacks. Timely detection of suspicious homograph domain names is an important step towards preventing sophisticated attacks, since this can prevent unaware users to access those homograph domains that actually carry malicious content. We therefore propose a structured approach to identify suspicious homograph domain names based not on use, but on characteristics of the domain name itself and its associated DNS records. To achieve this, we leverage the OpenINTEL active DNS measurement platform, which performs a daily snapshot of more than 65% of the DNS namespace. In this paper, we first extend the existing Unicode homoglyph tables (confusion tables). This allows us to detect on average 2.97 times homograph domains compared to existing tables. Our proactive detection of suspicious IDN homograph domains provides an early alert that would help both domain owners as well as security researchers in preventing IDN homograph abuse.

Title	A Case of Identity: Detection of Suspicious IDN Homograph Domains Using Active DNS Measurements
Authors	Ramin Yazdani, Olivier van der Toorn, and Anna Sperotto
Publication date	September 2020
Journal	2020 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)

Looking beyond the horizon: Thoughts on Proactive Detection of Threats

Tue, 04 Feb 2020 10:25:00 +0100

The fourth publication for the TIDE project. The FIRST talk (see here) has been extended into a journal paper for Digital Threats: Research and Practice (DTRAP). In this paper we argue that we, as a security community, should move towards proactive security. However, we shed light on both sides of the coin. We think the ‘optimal’ way is to combine the reactive and proactive methods, to make use of the best of both worlds.

The Internet exposes us to cyberthreats attacking information, services and the Internet infrastructure itself. Such attacks are typically detected in a reactive fashion. The downside of this approach is that alerts of an attack is issued as it is happening. In this paper weadvocate that the security community could benefit by complementing traditional reactive solutions with a proactive threat detectionapproach, as this would enable us to provide early warnings by analyzing and detecting threat indicators in actively collected data. Bydescribing three use cases from the DNS domain, we highlight the strengths and limitations of proactive threat detection and discusshow we could integrate those with existing solutions.

Slides of the presentation are available here: pdf

Title	Looking beyond the horizon: Thoughs on Proactive Detection of Threats
Authors	Olivier van der Toorn, Anna Sperotto
Publication date	March 2020
Journal	ACM Digital Threats: Research and Practice (DTRAP)

Star Wars

Sat, 31 Aug 2019 19:15:25 +0200

If you know me in real life you might know that I am a Star Wars fan (my nickname is a hint…).

Couple of days ago I stumbled across a nice video. In this YouTube video David Welch explains in great detail how the editing team of Star Wars saved the movie. I never knew how much difference was between what was actually shot and how we know the film today. David compares the arrangement of scenes as they were originally planned with how we know them. Along with why the order didn’t work, or what kind of effect it has on the story.

Today I’ve come across a Thumblr post suggesting an alternative theory as to who Rey actually is. Spoilers, the theory goes like this: Rey is a clone of Anakin. In the post Adam Serwer argues why this is possible and why it is more likely than Rey being the daughter of Luke or of Han.

Now I’m stoked for the last film in the latest trilogy…

Posters

Fri, 01 Mar 2019 15:05:57 +0200

Posters

The following posters were made during my time as a PhD student:

CSNG 2018

IMC 2018

SIGCOMM 2017

Ph.D. and Master Thesis

Fri, 13 Jul 2018 13:16:44 +0200

As of August the 30th I may call myself a Master of Science (MSc.). At that date I have successfully defended my thesis titled “Combating Snowshoe Spam with Fire”. People have often asked if ‘Fire’ is some kind of an acronym, it ain’t 😉

In the thesis we detail how DNS configurations may be used to track down snowshoe spam domains. Snowshoe spam spreads out the sending over a great number of hosts to reduce the volume per host, making the individual hosts harder to detect and blacklist.

During the time that I was working on my Master’s thesis I got the offer if I wanted to continue the work in a broader scope as a Ph.D. research. After talking with a few people and brewing about the idea, I accepted. Since September 15 I may call myself a Ph.D. student. The research project is titled TIDE, which stands for ‘Thread IDEntification Using Active DNS Data’. Last week I’ve created a website for the project, this can be found at https://www.tide-project.nl.

About

Fri, 13 Jul 2018 11:21:18 +0200

About me

This is me! (Well the photo is from ages ago…) I am a programmer at Storro. A small company based in Apeldoorn working on a secure file sharing and collaboration platform.

I used to be a Ph.D. student at the University of Twente. I worked on the proactive detection of malicious DNS domains. In October of 2022 I’ve successfully defended my thesis and can now call myself a doctor. My other interests lie with system administration, and playing guitar.

The blog

This page mainly serves as a page for me to blog about things I find interesting. Developments regarding my (professional) work can be found on the website of my Ph.D. project: Tide-Project

Contact

I am reachable via Matrix with the handle @olivier:oliviervandertoorn.nl.

Threat Identification Using Active DNS Measurements

Mon, 11 Jun 2018 00:00:00 +0200

The third publication for the TIDE project. Details more formally the research questions of this project.

The DNS is a core service for the Internet. Most uses of the DNS are benign, but some are malicious. Attackers often use a DNS do- main to enable an attack (e.g. DDoS attacks). Detection of these attacks often happens passively, but this leads to a reactive detection of attacks. However, registering and configuring a domain takes time. We want to pro-actively identify malicious domains during this time. Identifying ma- licious domains before they are used allows to pre-emptively stop an attack before it happens. We aim to accomplish this goal by analysing active DNS measurements. Via the analysis of active DNS measurements there is a window of opportunity between the registration time and the time of an attack, to identify a threat before it becomes an attack. Active DNS measurements allows us to analyse the configuration of a domain. Using the configuration of a domain we can predict if it will be used for malicious intent. Machine Learning (ML) is often used to process large datasets, because it is efficient and dynamic. This is the reason we want to use ML for the detection of malicious domains. Since our results are predictive in nature, methodology for validation of our results need to be developed. Because, at the time of the detection no ground truth is (yet) available.

Slides of the presentation are available here: pdf

Title	Threat Identification Using Active DNS Measurements
Authors	Olivier van der Toorn, Anna Sperotto
Publication date	2018/6/4
Conference	AIMS 2018

Melting the Snow: Using Active DNS Measurements to Detect Snowshoe Spam Domains

Thu, 03 May 2018 00:00:00 +0200

The second publication for the TIDE project. It has received the Best Paper Award at NOMS 2018.

Snowshoe spam is a type of spam which is notoriously hard to detect. Differently from regular spam, snowshoe spammers distribute the volume among many hosts, in order to make detection harder. To be successful, however spammers need to appear as legitimate as possible, for example, by adopting email best practice like Sender Policy Framework (SPF). This requires spammers to register and configure legitimate DNS domains. Previous studies uses DNS data to detect spam. However, this often happens based on passive DNS data. In this paper we take a different approach. We make use of active DNS measurements, covering more than 60% of the namespace, in combination with machine learning to identify malicious domains crafted for snowshoe spam. Our results show that we are able to detect snowshoe spam domains with a precision of more than 93%. Also, we are able to detect a subset of the malicious domain 2?104 days earlier than the spam reputation systems (blacklists) currently in use, which suggest our method can give us a time advantage in the fight against spam. In a real-life scenario, we have shown that our results allow spam filter operators to block spam that would otherwise bypass their mail filter. A Realtime Blackhole List (RBL) based on our approach is currently deployed in the operational network of a major Dutch ISP.

Slides of the presentation are available here: pdf

Title	Melting the Snow: Using Active DNS Measurements to Detect Snowshoe Spam Domains
Authors	Olivier van der Toorn, Roland van Rijswijk-Deij, Bart Geesink, Anna Sperotto
Publication date	2018/4/23
Conference	NOMS 2018

Best Paper Award at NOMS 2018

Tue, 01 May 2018 00:00:00 +0200

TIDE was present at the Network Operations and Management Symposium (NOMS 2018) conference in Taipei, Taiwan. Olivier was there to present “Melting the Snow: Detecting Snowshoe Spam Domains Using Active DNS Measurements”.

NOMS 2018 was held in Taipei, Taiwan, from the 23rd till the 27th of April. NOMS has been held in every even-numbered year since 1988. This was the 30th anniversary of NOMS.

Our work was very well received at the conference. So well, in fact, that they gave us the the Best Paper Award!

The pre-print of the paper can be found here: pdf — bibtex

And the presentation slides can be found here: pdf

Thanks to Oliver Festor and Roland van Rijswijk-Deij for the pictures.

Virtual Reality Browser

Wed, 01 Mar 2017 01:00:00 +0200

For the last couple of weeks I have been developing a Virtual Reality Browser. In true Linux fashion, I was not satisfied with the VR Browsers available in the Google Play Store, so I developed my own. Yes, I know, to really adhere to the fashion I should fork an existing one…

It is, has been, an interesting experience. This is mainly due to me not having any experience in Android app development, at all. Having no OpenGL experience did not really help either.

My idea was to have a plane some distance away from the camera (the eyes of the user), onto which a browser (WebView) is projected. The blog post of Felix Jones helped me a lot of getting to my destination. In his blog post he explains how to render any View onto an OpenGL surface.

Having that as a starting point I started hacking away. Currently I am in a state where the VR Browser works. Sure there are still a couple of bugs which make the application crash. But on the other side. Basic browsing can be done. It even supports multiple tabs.

The way to control the browser happens exclusively through an external keyboard. I made a number of keyboard shortcuts to do everything. From the usual browser shortcuts to moving the camera. There is also a keyboard shortcut for simulating a tap on the display.

Hopefully this application will come to the Google Play Store soon.

Fosdem 2017 Part2

Wed, 01 Mar 2017 00:00:00 +0200

Bit of a late post, but better late than never. [FOSDEM 2017][1] was a nice time, as usual. Didn’t get to every talk I wanted to go to, but did see many interesting talks.Looking at the FOSDEM 2017 post, the first talk I had selected and actually attended was ‘Introduction to A-Frame’. A nice introductory talk about A-Frame. A-Frame being a framework for web based Virtual Reality applications. The inspector part is really neat. It allows one to edit a scene as if you are in a 3D modeler.

Another really noteworthy talk I attended was the ‘Debugging Hung Python Processes with GDB’. The idea was that you can use GDB to debug Python processes if you use the CPython interpreter. Given you installed packages to aid GDB in Python debugging, it would look like you used the Python debugger (PDB) instead. I should experiment with this myself.

In terms of goodies, FOSDEM 2017 was fruitful. I picked up three t-shirts for myself and one for my girlfriend. CZ.NIC had a booth promoting their Omnia Turris router. They had a challenge where you had to put your name on a display. This display was attached to the Omnia Turris. The solution was to connect to the access point, browse to the gateway and submit your name. As a reward you’d get a t-shirt. They were out of male medium and large t-shirts, so I got one for my girlfriend instead. The three shirts for myself were, a FOSDEM one, a Foreman one and a KDE one.

All in all, it was a nice time! [1]: /post/fosdem-2017/

FOSDEM 2017

Fri, 03 Feb 2017 00:00:00 +0200

This is the third year in a row that we are going to FOSDEM. Each year it was blast, and hopfully this one will be too.

Of course, the event starts with many nice Belgium beers. Looking forward to that.The talks on saturday and sunday which I will most likely go to are:

The Veripeditus AR Game Framework, by Dominik George / Eike Jesinghaus

Recently I have formed an intrest in Virtual Reality. Purchased a Google Cardboard. Hope to see a bit more uses for this kind of technology.
Everything You Always Wanted to Know About “Hello, World”, by Brooks Davis

A talk about the more internals of C.
Does your configuration code smell?, by Tushar Sharm

Writting better configuration code is allways beter 😉
WebRender, the next generation graphics engine by Mozilla research, by Nicolas Silva

Don’t really know anything about WebRendering, good oppertunity to learn something new 🙂
Introduction to A-Frame, by Eugenio Petullà

Again, the VR stuff. But this time on the Web.
Computer games – not as easy as it looks, by Steven Goodwin

The summary seemed interesting.
It’s time for datetime, by Mario Corchero

Datetime is often used. Perhaps I can learn a few new tricks.
Hacking midi devices with StepPy, by Yann Gravrand

Sequencers and Python, should be fun.
Debugging Hung Python Processes with GDB, by Brian Bouterse

Usefull skills to have.
Desktop security, keeping the key to the castle safe, by Mickael Scherer

Securing desktops… should be interesting.
WTF my container just spawned a shell, by Jorge Salamero

Might be interesting.

Well, that is just a selection. Honestly I have not looked at overlaps (allways a problem). We’ll see where I end up.

Maybe I’ll see you there?

Bachelor Thesis

Sun, 05 Apr 2015 00:00:00 +0200

My Bachelor assignment was the perfect way of deciding at which chair I wanted to follow the Electrical Engineering Master. I was interested in both the Design and Analysis of Communication Systems (DACS) chair and the Telecommunication Engineering (TE) chair. Ultimately I decided to do my Bachelor assignment at the DACS chair. So I met with all the AIO’s working at DACS to see what kind of assignments they could offer me. All of the assignments were on state of the art topics. Cloud-networks for mobile providers, dimensioning link-capcity, etc. I chose to work with Rick Hofstede, his assignment was about HTTP(S) intrusion detection. His previous research was in the field of SSH intrusion detection, and he wanted to see if the same could be done with HTTP(S).

What does ‘HTTPS Intrusion Detection’ mean? Even though the term HTTP(S) is clear for most of you I am going to give an analogy anyways, as it will help explain other things too. Suppose that instead of visiting Web sites with your browser you want to physically hold the Web pages in your hand. You send a post-card to Scintilla requesting their home-page. The Web server reads this request prints the page puts it in a package and sends it to your door through PostNL. You open the package and view the Web page. In this scenario PostNL can be compared to HTTP. For HTTPS it can be imagined that the package is given a lock of which only the sender and receiver have the key. With ‘intrusion’ most people think of a burglar breaking into their home and stealing all their valuable items. For the Web this is quite similar. But instead of a home there is the back end, or a control-panel, of a Web site. And instead of a door with a lock there is an authentication mechanism. In my research I have looked at three authentication mechanisms. HTTP Basic Authentication (BA), Form-based Authentication (FA) and XMLRPC. There are several ways an attacker can try to gain unauthorized to such a back end, the one we have researched is the brute-force attacks. A brute-force attack is simply trying every login combination of username and password you can think of. These brute-force attacks are usually based on a list of commonly used login credentials called a dictionary, hence these attacks are also known as dictionary attacks.

Figure 1: Attack phases

Dictionary attacks typically feature three phases. The first phase is the ‘scan phase’, here an attacker scans the network for the targeted services. The second phase is the ‘brute-force phase’, this is the phase where all the login credentials are tried. This can end in two ways. Either no valid credentials are found and the attack is ceased. Or the last phase is seen, the ‘compromise’ phase, here the attacker has gained entry to the back end and is, for example, able to upload illegal content. Brute-force attacks are usually detected by analysing access logs, if they are detected at all. This host-based approach is hardly scalable in larger networks, since access to the logs is required. Besides the host-based approach, a network-based approach can be taken. This approach can be divided into two categories, namely packet-based and flow-based. To explain these two categories we take another look at our PostNL analogy of earlier. Packet-based intrusion detection systems can be seen as systems that open each and every packet that passes by to analyse their contents for malicious traffic. As you’ve likely realized if the packages are encrypted, if a lock is added, the IDS is no longer able to open the package and analyse its contents. The flow-based approach does not face this problem as it looks at packet headers, not their payload. A flow can be seen as the label that is attached to each packet. It lists the sender and destination. The weight of the package. If the package is part of a sequence it lists how long the sequence is, etc. This analogy is not very accurate, but it gives you an idea of what a flow is. Analysing the traffic generated by dictionary attack tools allowed us to develop signatures. These signatures, as shown in Figure X, can be used to detect dictionary attack from flow data. As can be seen there are two ranges defined, the packets per flow (PPF) and bytes per flow (BPF).

During my Bachelor assignment much effort was put in developing a flow-based prototype IDS. This prototype uses the signatures we have developed to detect dictionary attacks from given flow data. It detects attacks in three stages. First, a preselection stage, here the data is filtered to generate a list of source and destination IP address tuples with at least one flow matching at least one flow. Second, the detection stage, this is where the detection algorithm comes in. Every flow between the preselected IP address tuples are checked against the signatures. As the signatures defines different ranges there are also different modes of operation. Either only the PPF, or BPF, is used or both the PPF and BPF are used for the signature matching. If a tuple shows a consecutive number of flows higher than a given threshold it is marked as being an attack. And thirdly, the signature matching stage. This stage is necessary as there can be multiple signatures used in the detection stage. The basis of the signature matching algorithm finds its roots in the field of digital communication, namely the signal space concepts, where bits are mapped to a signal space to determine if a one or a zero was sent and received. Instead of using bits in a constellation diagram, we user the number of PPF and BPF on the axes of an imaginative constellation diagram, and the Pythagorean theorem for finding the signature that is closest to the analyzed traffic.

Figure 2: Detection accuracy under different flow record thresholds

We are number one! That was rather difficult seeing that we were the only one around. But in all seriousness, accuracies of around 100% are achievable with the prototype. However we must acknowledge that there are false positives, normal traffic that is marked as being an attack, these false positives are mainly caused by (legitimate) automated traffic, such as RSS parsers, Web calendar fetchers and SPAM being posted on blogs. This gives a false positive of around 10%.

The results of this assignment were documented in a conference paper. In there we have presented the first steps in the field of flow-based HTTP(S) intrusion detection. We have shown that the developed prototype in combination with the signatures is able of achieving accuracies of around 100%. However there are false positives, these are mainly caused by legitimate automatic traffic. We realize these types of traffic can be of great importance to Web site owners, as they often rely on search engine rankings for their income, for example. Further investigation of this traffic will therefore be part of our future work. In talks with Antagonist, we have learned that a system as presented in the paper may prove very useful. For example, it could be integrated with an automated system that blocks attackers based on detection results of our IDS. Requests from blocked IP addresses could be forwarded to a static landing page, from which one can choose to be unblocked. Since such behaviour is not understood by attack tools, humans can easily be unblocked while automated attacks are mitigated.

As this is only the first step in intrusion detection against HTTPS, there remains a lot of work to be done. If you are interested in continuing were I left off, contact Rick Hofstede from the DACS chair.

Linux Volume Manager and Kernel-Based Virtual Machines

Tue, 22 Apr 2014 00:00:00 +0200

Few months ago I started experimenting with Linux Volume Manager (LVM). Being encouraged by friends who where already using LVM and showing me how easy it could be to resize partitions, make new ones, etc.

My desktop needed reinstalling anyhow, so I figured why not try out LVM right away. Since I had no experience with LVM I let the (K)Ubuntu installer set up LVM for me. It made a /boot as the first partition with a ext2 filesystem. The rest of the disk was an LVM physical volume. This pv was added to the ‘star-destroyer-vg’ volume group. In this vg there where two logical volume created, a root lv and a swap lv.

Usually I have a separate partition for /home. So I went ahead and added it with lvcreate. Then mounted it to /mnt, copied the current contents of /home to /mnt. Removed the contents of /home and mounted the home lv to /home. Lastly I added the mount to fstab.

Then the moment of truth, reboot and see if everything works.

Rebooting…. Logging in…

mount|grep home-lv

Whoo there is output 😀

After a couple of days I tried a live-resize. I can’t recall what partition I resized at the time, but it is still an awesome feature to just say:

lvresize -L+5G <path-to-lv> && resize2fs <path-to-lv>

And your partition is 5Gb larger, without unmounting or booting in to live-disks :D.

Then I started to look at Kernel-based Virtual Machines. At the time I used Virtual Box for my VM needs. Phoronix compared KVM to Virtual Box, Xen and Bare Metal. It was concluded that KVM was not much below Bare Metal in terms of performance. Installed lib-virt and virt-manager on my desktop. Since I’m part of the Kubuntu Testers Team the first install I did was a Trusty install.

Lovely thing to me was that I could just hand the virtual machine a logical volume as disk. Also the performance of these virtual machines was quite fine. They seemed to be faster than the Virtual Box VMs, though that might just be me wanting KVM to be faster.

I liked KVM so much I installed in onto my server. The host that is serving you this page is actually a virtual machine.

Backlinks are bad

Mon, 01 Jan 0001 00:00:00 +0000

Backlinks are bad because they do not provide any additional information. Rather they take away in cognitive performance because there is another link you might follow. Rather links should come from a natural flow, because the note builds on a previous note, for example. Or to include an idea from another note.

I’ve come to believe that linking keywords within the idea to other notes is a better approach than a list at the bottom linking back and forth.

Sources

https://zettelkasten.de/posts/backlinks-are-bad-links/

How to Link Notes

Mon, 01 Jan 0001 00:00:00 +0000

Notes should be linked, as is the basis of the Zettelkasten idea. However, to effectively link notes try the following:

Link to a more general idea.

Link to a more specific idea.

Link to a related topic.

See “tk notes” for when you don’t have note ready to link to.

Above is how we used to think about linking notes together. In the meantime we have learned that there is some truth to this manner, but having these as a list at the bottom of a note doesn’t work.

Linking notes in the body of the note itself works a lot better. This makes clear what you are linking to and puts some context behind the link.

Sources

https://medium.com/@philhoutz/make-your-notes-work-for-you-the-secret-sauce-of-zettelkasten-cb901a3cce00

Tags in Zettelkasten

Mon, 01 Jan 0001 00:00:00 +0000

In his Zettelkasten Luhman used tags sparringly, in this video explains that Luhman used tags more as doors into the Zettelkasten, rather than as a way of grouping things together. Additionally, it makes automatic searching for keywords harder and forces you to look through your ideas manually. Which improves the linking of notes.

Sources

https://www.youtube.com/watch?v=7TnUNN39NBU&list=WL&index=14

The Zettelkasten principles

Mon, 01 Jan 0001 00:00:00 +0000

There are a few principles attached to a Zettelkasten:

The principle of atomicity: A note should contain one idea, and one idea only. The reason for this is that you want to be able to link to an idea, and not some idea hidden on a card somewhere in the middle.
The principle of autonomy: Each note should be self-contained and comprehensible. This includes autonomy from other notes, but also from the sources. As the note in the Zettelkasten may outlive these. You still want the note to be readable in that case.
Always link your notes: Make sure to link notes to existing notes. According to Luhmann “A note that is not connected to the network will be lost, will be forgotten by the Zettelkasten”.
Explain why you’re linking notes: Explain why two notes are linked together. A future self needs to be able to understand the link.
Use your own words: In order to ensure that you understand the idea you need to write ideas in your own words.
Keep references: Add references to where the idea is coming from. Preventing plagiarism and making it easy to find the source again, when needed.
Add your own thoughts to the Zettelkasten: Add your own thoughts, keeping in mind the above principles.
Don’t worry about structure: Don’t worry about the categories or tags too much. The organization develops organically, as the kasten grows.
Add connection notes: These notes describe the relationship between notes.
Add outline notes: These denote themes, an outline note contains a sequence of links to other notes.
Never delete: Instead of deleting notes, create a new one detailing where the previous one is wrong. This strengthens the reasoning. Deals with hindsight bias and allows to revisit old ideas.
Add notes without fear: This might be the most important principle for a new Zettelkasten. There is never “too much” information in a Zettelkasten. At worst, the note won’t be used, it can’t break the system.

Sources

https://writingcooperative.com/zettelkasten-how-one-german-scholar-was-so-freakishly-productive-997e4e0ca125

When should you start a new note?

Mon, 01 Jan 0001 00:00:00 +0000

The following image shows when you should start a new note for your slipbox. And what to do if there are similar notes:

Start by finding similar notes, if there aren’t any make a new note. If there are notes, but not related, make a new note.

Notes related to the topic exist, if these do not fit the idea create a new note but adhere to the keywords.

A few notes roughly match: creating a new zettel is a bit too much. In this case it is useful to compile the existing knowledge. Comment on the compilation and make links in both directions. Ignore the last line, backlinking is bad

When an exact match exists, you can simply extend what is already there.

This note is largely based on this documentation.

Sources

https://zettelkasten.de/posts/when-start-new-note/

Zettelkasten

Mon, 01 Jan 0001 00:00:00 +0000

#notes

A Zettelkasten is said to be a second brain. Through the linking of notes the system may surprise you in insightful ways.

What is a zettelkasten?

The word can be split into two ‘zettel’ and ‘kasten’. Zettel meaning a note and kasten simply box. So, a zettelkasten is a box of ideas. But what makes this box special/useful?

Ideas behind zettelkasten

Rather than writing notes down in categorized notebooks the notes should have a free-flowing category. This Medium article has some nice visualizations of this. Where the author starts with the ’normal’ system of notebooks, a set of dots (representing the notes) cast in concrete.

If the notes were kept on index cards it would more look like notes free-floating in air. Yet, impossible to track how notes relates to one another.

Putting these in rigid folders does organize them, but makes it too static. What if a note should be in two folders at the same time?

Tagging notes helps here. As notes can have multiple tags they can be part of multiple ‘categories’. See this why tagging should be used sparsely.

However, Luhmann went further, next to tagging notes he also linked notes together. Meaning that notes are not just grouped together, but also their connections are explicitly tracked. Creating a web of ideas. Additionally, it shows grouped topics. This means that categories naturally develop and are not forced upon notes beforehand.

Surprising factor

Claude Shannon realized that there is more information in a surprising message than a non-surprising message.

This relates to the Zettelkasten in the following way: Suppose the Zettelkasten is filled to the brim with forgotten ideas. Then with a question in mind you search the Zettelkasten stumbling upon all these ideas, surprising you.

The process

The process for writing a new note is explained here. The ID of a (new) note should follow these guidelines The principles of a Zettelkasten do need to be kept in mind.

What goes into the Zettelkasten?

The main thing that goes into a Zettelkasten are, ofcourse, notes. However, I think it is valuable to add quotes and thoughts too. It is important to detail the source though.

Sources

https://writingcooperative.com/zettelkasten-how-one-german-scholar-was-so-freakishly-productive-997e4e0ca125