Sunday, July 31, 2011

OSDEV Series - Episode 2.1: Short recap

Rings

Rings offer a protection layer for programs. They allow certain levels of resource access to processes, so it keeps bad programs from messing things up. There are a total of 4 rings in most common architectures. However, many architectures have only two rings (e.g. x86_64), corresponding to ring 0 and 3 in this description.

Ring 0:
This is kernel mode or supervisor mode. This level has the least protection, and the most access to resources. When starting up, the OS runs in this mode unless it switches out. Interrupt handlers run in this mode.

Rings 1 and 2:
These rings are mostly used for device drivers. They offer more protection, but not as much as ring 3.

Ring 3:
This is the ring that most OS's use for applications. This ring is also called Userland, or Userspace. It has the most protection and the least resource access.

Most operating systems use only Ring 0 and 3. This is because rings 1 and 2 are unneeded, as device drivers can run in either ring.
Sometimes applications need access to resources that their ring wont allow. If they try to access them, a General Protection Fault (int 13) will be triggered, and the application shutdown. The application must interface with the kernel somehow, and mostly this is done with System Calls.



Kernel designs

Microkernels:
A microkernel only implements the basic services needed for applications, such as memory management, or multitasking. Other services implemented in user space, also known as servers, offer additional functionality that the kernel would offer in other designs. Such services may include networking, file systems etc.

Pros: Lightweight kernel, easy to maintain.
Cons: A large number of system calls is required for servers, and this costs a lot of performance.


Monolithic kernels:
All the services are run in kernel space

Pros:
Rich and powerful hardware access
Easier to implement than other designs

Cons:
All the services must run smoothly, otherwise the entire system could crash
Harder to maintain, since the kernel is pretty large

Examples of kernels:
UNIX (and UNIX-like)
DOS


Other kernel designs:

  • Hybrid kernels: A compromise between the two big designs. The most popular example of a hybrid kernel is Windows NT.
  • Nanokernels: Very small microkernels.
  • Exokernels: As little abstraction as possible. This gives very much power to applications. However, it is not a very deeply researched design, and implementation can be very difficult.



Abstraction

In computer science, abstraction is the process by which data and programs are defined with a representation similar to its meaning (semantics), while hiding away the implementation details. In other words, it is the method of hiding away ugly stuff behind really nice functions that programmers can use.

To exemplify, let's look at a typical file read operation. The application reads a file from the computer, by calling an fread() function from the system library. The fread() function itself, makes a system call, asking the kernel to read the file. The kernel goes through the virtual file system to find the file. First it finds the device, such as the hard disk or memory card, then reads the actual sectors, and then parses the file system. The sector read function may be calling the firmware present in the device.

All these steps are in fact abstraction layers. A read operation starts at top level, where the application is situated, and goes all the way down to hardware level.

OSDEV Series - Episode 2.1 - Rings and kernel designs

Section 2: The basics
Episode 1: 
Rings and kernel designs


Covered topics:

Theory: Rings, kernel designs, abstraction
Practice: Coding the entry point in assembly, creating linker and shell scripts for compiling

Project files: http://dl.dropbox.com/u/24832466/OSDEV/MyOS-2.1.zip

Sunday, May 8, 2011

OSDEV Series - Episode 1.3: Short recap

The boot process:
When you turn on your computer, several things happen:
1. All the hardware is powered
2. When power is good, the processor starts, executing firmware
3. The firmware (also known as the BIOS = Basic Input/Output System) runs system checks (POST = Power On Self Test)
4. The BIOS looks for bootable devices, such as diskettes, hard disks, or DVDs
5. When a bootable device is found, the first sector is loaded in memory, and executed.
Inside the first sector, there is a part of our operating system. Now the operating system has full control over the computer.


Components of an operating system:
1. Bootloader
Description: It is a small program, that is located in the boot sector.
Role: Does hardware detection, prepares the computer for operating mode, and executes the kernel. Some bootloaders also give the possibility of booting multiple kernels, and show a menu to the user.
Why it exists: The space in the boot sector is very limited. For example, on a floppy disk there are only 512 bytes.


2. Kernel
Description: The core of all operating systems.
Role: It does essential stuff, such as:
- Hardware input and output
- Memory management
- Makes executing programs possible

3. Programs
Special programs:
Drivers: have special privileges in comparison with normal programs.
Shell: comes with most operating systems. It is like an 'add-on' that allows user interaction.
Normal programs:
Examples: text editor, browser, videogames etc.



Setting up GRUB:
There are 2 floppy images needed:
1. myos.img - will contain the operating system
2. aux.img - auxiliary image, used to install GRUB on image #1.

The following commands will be executed in the Linux terminal, or Cygwin.


Step 1: Generating the images:
dd bs=512 count=2880 if=/dev/zero of=myos.img
dd bs=512 count=2880 if=/dev/zero of=aux.img

Step 2: Format and mount myos.img:
Attention: Linux users need administrator priviledges for the following instructions. This is done by placing sudo before the commands, in Debian based distributions.
Format the image: mkfs.msdos myos.img
Create a directory: mkdir /media/floppy1
Mount the image: mount -o loop myos.img /media/floppy1

Step 3: Create configuration file for GRUB
Create a new file called menu.cfg, and using your favorite text editor, write the following lines:

  default 0
  timeout 0
  hiddenmenu

  title MyOS
  root (fd0)
  kernel /kernel.bin
  boot


Step 4: Put all necessary files on the mounted floppy image
The floppy disk will contain the following file structure:
/boot/stage1
/boot/stage2
/boot/menu.cfg
kernel.bin


kernel.bin is the kernel of the operating system. We will create this file in the next episode.

Now we will copy all the files on the disk:
Create boot folder: mkdir /media/floppy1/boot
Copy stage files: cp stage? /media/floppy1/boot
Copy configuration file: cp menu.cfg /media/floppy1/boot

Step 5: Unmount the image
Unmount it: umount /media/floppy1
Delete folder: rm -r /media/floppy1

Step 6: Copy stage1 and stage2 directly inside the auxiliary image
stage1: dd bs=512 count=1 if=stage1 of=aux.img conv=notrunc
stage2: dd bs=512 seek=1 if=stage2 of=aux.img conv=notrunc

Step 7: Install GRUB on disk #1 using the auxiliary image
Using VirtualBox, we will start a virtual machine, with the auxiliary image inserted.
After the GRUB command line appears, insert myos.img, and type the following command:
install (fd0)/boot/stage1 (fd0) (fd0)/boot/stage2 (fd0)/boot/menu.cfg




Finally, I organized the files a little bit, and this is the new folder structure:
Boot/grub/aux.img
Boot/grub/menu.cfg

Boot/grub/stage1
Boot/grub/stage2
Boot/myos.img

OSDEV Series - Episode 1.3 - Babysteps

Section 1: Introduction
Episode 3: Babysteps




Covered topics in this episode:

Theory: the boot process, components of an operating system
Practice: setting up GRUB

In the next episode, we start writing our kernel, and we will learn more about kernel designs.

Saturday, March 26, 2011

OSDEV Series - Episode 1.2: Software list

Here is a list with all the software used, and how to get it.

Name: Cygwin
Operating system: Windows
Download source: http://www.cygwin.com/
Description: Linux like shell that runs in windows, also includes compiler, linker and many other useful tools.

Name: GCC Toolchain
Operating system: Linux
Download source: Should be already installed on most Linux distributions. Can be installed using** sudo apt-get install gcc-core gcc-g++
Description: C and C++ compiler.

Name: NASM (NetWide Assembler)
Operating system: Windows*, Linux
Download source: ** sudo apt-get install nasm
Description: x86 assembler.

Name: Notepad2
Operating system: Windows
Download source: http://www.flos-freeware.ch/notepad2.html
Description: Simple text editor with syntax highlighting.

Name: Notepad++
Operating system: Windows
Download source: http://notepad-plus-plus.org/
Description: Advanced source code editor, has many more features than Notepad2.

Name: GEdit
Operating system: Linux
Download source: Already installed on Gnome distributions.
Description: Simple text editor with syntax highlighting.

Name: Kate
Operating system: Linux
Download source: Already installed on KDE distributions.
Description: Simple text editor with syntax highlighting.

Name: HxD
Operating system: Windows
Download source: http://mh-nexus.de/en/hxd/
Description: Powerful hex editor.

Name: GNOME Hexadecimal Editor
Operating system: Linux
Download source: ** sudo apt-get install ghex
Description: Hex editor for Gnome.

Name: VirtualBox
Operating system: Windows, Linux
Download source: http://www.virtualbox.org/wiki/Downloads
Description: Virtualization software.

Name: Qemu
Operating system: Linux
Download source: ** sudo apt-get install qemu
Description: Virtualization software for linux.

Name: Bochs
Operating system: Windows, Linux
Download source:
Linux:** sudo apt-get install bochs
Windows: http://bochs.sourceforge.net/
Description: Operating system debugger.

Notes:
* On Windows, Nasm should be installed using Cygwin, for full compatibility with the tutorial.
** The apt-get command is only available on Debian/Ubuntu based distributions. On other distributions, there may be Yum or other package managers installed.

OSDEV Series - Episode 1.2 - Setting up the environment

Section 1: Introduction
Episode 2: Setting up the environment




This is the second episode in the series, and this time we will set up our environment for operating system development.
Windows starts at 0:00, Linux at 5:00.

In the next episode, we will install the GRUB bootloader on a virtual floppy image, and we will cover some basic theory about operating systems.

Notice

The blog has moved to a new address: http://luxsystem.blogspot.com

This is due to the CTA project being canceled, and a new project called Lux Operating System has started. There are many reasons for this change, but however a lot from the old project will be used in the new one, but some parts will be rewritten, with increased efficiency and speed.