Andreas Christoforou

Building WebKit

2023-11-29T17:05:54+00:00

Building WebKit

WebKit is the web browser engine used by Safari, Mail, the App Store, and various other applications on macOS and iOS.

To build WebKit, you must have Xcode installed from the App Store. Then, run the following command in Terminal to install the Xcode Command Line Tools:

xcode-select --install

Cloning the WebKit Project

Open Terminal on your macOS. Use the following command to clone the WebKit repository from GitHub:

git clone https://github.com/WebKit/WebKit.git WebKit
cd Webkit

Building WebKit

To build WebKit, run the following command in Terminal:

./Tools/Scripts/build-webkit

Building WebKit with Debug Flag and ccache

To build WebKit with debugging information and improve recompile times, use the following command in Terminal:

./Tools/Scripts/build-webkit --debug --use-ccache

The compilation will take about 20 minutes to complete.

Running WebKit in Safari

./Tools/Scripts/run-safari --debug

Running the MiniBrowser

The MiniBrowser is a lightweight tool for testing and debugging your WebKit build. Here’s how to run it:

./Tools/Scripts/run-minibrowser --debug

References

https://webkit.org/building-webkit/

Building Chromium for Android

2020-01-29T17:32:51+00:00

How to build chromium for Android.

Download Tools

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export PATH="$PATH:/path/to/depot_tools"

Get Chromium Project

mkdir ~/chromium && cd ~/chromium
fetch --nohooks android
 
cd src
echo "target_os = [ 'linux', 'android' ]" >> ../.gclient
gclient sync
build/install-build-deps.sh
gclient runhooks

Fetch the latest tags for Android and replace VERSION with the correct value.

git fetch --tags
git checkout VERSION
gclient sync -D --with_branch_heads --with_tags --jobs 32

Install dependencies

build/install-build-deps.sh

Setting up the build

Create args.gn

mkdir -p out/Default
	
cat <<EOF > out/Default/args.gn
target_os = "android"
target_cpu = "arm64"
android_channel = "stable"
android_default_version_name = "131.0.6778.39"
android_default_version_code = "677803900"
ext_version_enabled = true
ext_version_increment = "0"
is_component_build = false
is_debug = false
is_official_build = true
symbol_level = 1
disable_fieldtrial_testing_config = true
dfmify_dev_ui = false
ffmpeg_branding = "Chrome"
proprietary_codecs = true
is_cfi = true
use_cfi_cast = true
use_relative_vtables_abi = false
include_both_v8_snapshots = false
enable_vr = false
enable_arcore = false
enable_openxr = false
enable_cardboard = false
enable_remoting = false
enable_reporting = false
EOF

If you want to build the chromium for debugging or to fuzz with libfuzzer change or add the following

is_asan = true
is_hwasan = true
is_msan = true
is_lsan = true
is_tsan = true
msan_track_origins = 2
is_ubsan_no_recover = false
is_ubsan_security = true
is_debug = true
dcheck_always_on = true
is_java_debug = true
is_component_build = true
use_libfuzzer = true
use_fuzzilli = true

GN build configuration

gn gen out/Default

Build

The following command is used to build Trichrome for Android minSdkVersion=29.

autoninja -C out/Default trichrome_chrome_bundle

The following command is used to build Monochrome, which provides both Chromium and the WebView for Android minSdkVersion=26

autoninja -C out/Default monochrome_public_bundle

for devices with minSdkVersion=26 and for local development (to avoid building WebView).

autoninja -C out/Default chrome_public_apk

Install to Device

out/Default/bin/trichrome_chrome_bundle install

Running a test

out/Default/bin/trichrome_chrome_bundle launch https://google.com

Logging and debugging

You can see the logs with adb logcat when you will install the chrome apk or with

out/Default/bin/trichrome_chrome_bundle logcat [-v]

Debugging Java and c/c++ code

C/C++ Debugger

out/Default/bin/trichrome_chrome_bundle gdb

out/Default/bin/trichrome_chrome_bundle lldb

Java Debugger

out/Default/bin/trichrome_chrome_bundle run --wait-for-java-debugger

Symbolizing Crash Stacks and Tombstones (C++)

build/android/tombstones.py --output-directory out/Default
adb logcat -d | third_party/android_platform/development/scripts/stack --output-directory out/Default

Memory Tagging

2019-04-07T17:30:54+00:00

ARM Memory Tagging

Memory tagging is a tag assigned to each memory allocation, so all accesses to memory must be via a pointer with correct tag, if the tag is wrong the Operating System can report it to user or note the process in which occured.

Furthermore memory tagging can be used for debugging in development to detect memory errors.

Allocations are done with align allocations by 16-bytes and choose a 4-bit tag which same tag is set to the corresponding pointer and memory. Deallocations are working with re-tag the memory with a different tag.

Example

char *p = new char[20];

20 bytes needs to allocated, memory allocator aligns 32 bytes, 16 bytes will be used for tag granularity, the tag will be on the 4 most significant bits of pointer and memory, so when the attacker will try to dereferencing pointer p at position 32, tag values will not be correct between pointer and memory and overflow will be caught.

Memory Tagging Extension instructions

ARM implementated the memory tagging hardware support with tag size of 4-bit and tag granularity of 16-bytes and they have introduced some of the following new instructions

IRG - Insert Random Tag inserts a random logical address tag into the address in the first source register and writes the result to the destination register.

IRG Xd|SP , Xn|SP; // irg  x0, sp;

CMPP - Compare with TAG subtracts the 56-bit address held in the second source register from the 56-bit address held in the first source register, updates the condition flags based on the result of the subtraction, and discards the result.

CMPP <Xn|SP>, <Xm|SP>

ADDG - Add with Tag adds an immediate value scaled by the Tag granule to the address in the source register, modifies the Logical Address Tag of the address using an immediate value, and writes the result to the destination register. Tags specified in GCR_EL1 system register.

ADDG <Xd|SP>, <Xn|SP>, #<uimm6>, #<uimm4>; // addg  x19, x0, #16, #1

STG - Store Allocation Tag stores an Allocation Tag to memory. The address used for the store is calculated from the base register and an immediate signed offset scaled by the Tag granule. The Allocation Tag is calculated from the Logical Address Tag in the source register.

STG [<Xn|SP>], #<simm>; // stg  x0, [x0]

LDG - Load Allocation Tag loads an Allocation Tag from a memory address, generates a Logical Address Tag from the Allocation Tag and merges it into the destination register. The address used for the load is calculated from the base register and an immediate signed offset scaled by the Tag granule.

LDG <Xt>, [<Xn|SP>{, #<simm>}]

ST2G - Store Allocation Tags stores an Allocation Tag to two Tag granules of memory. The address used for the store is calculated from the base register and an immediate signed offset scaled by the Tag granule. The Allocation Tag is calculated from the Logical Address Tag in the source register.

ST2G <Xt|SP>, [<Xn|SP>], #<simm>; // st2g  x8, [sp], #32

New System Registers for tagging

TCO - Tag Check Override When ARMv8.5-MemTag is implemented, this register allows tag checks to be disabled globally.
TFSRE0_EL1 - Tag Fail Status Register Holds accumulated Tag Check Fails occurring in EL0 which are not taken precisely.
TFSR_EL1 - Tag Fail Status Register Holds accumulated Tag Check Fails occurring in EL1 which are not taken precisely.
TFSR_EL2 - Tag Fail Status Register Holds accumulated Tag Check Fails occurring in EL2 which are not taken precisely.
TFSR_EL3 - Tag Fail Status Register Holds accumulated Tag Check Fails occurring in EL3 which are not taken precisely.
RGSR_EL1 - Random Allocation Tag Seed Register.
GCR_EL1 - Tag Control Register.

Compiler Memory Tagging

MTE Example

// Compiled with following options -march=armv8.5-a+memtag -fsanitize=memtag 
// llvm-objdump -d test

0000000100007f04 _main:
100007f04: 7f 23 03 d5                 	hint #27
100007f08: ff 03 01 d1                 	sub	sp, sp, #64
100007f0c: fd 7b 03 a9                 	stp	x29, x30, [sp, #48]
100007f10: fd c3 00 91                 	add	x29, sp, #48
100007f14: 08 00 00 b0                 	adrp	x8, #4096
100007f18: 08 09 40 f9                 	ldr	x8, [x8, #16]
100007f1c: 08 01 40 f9                 	ldr	x8, [x8]
100007f20: a8 83 1f f8                 	stur	x8, [x29, #-8]
100007f24: 08 00 80 d2                 	mov	x8, #0
100007f28: e8 13 c8 9a                 	<unknown>
100007f2c: 08 01 81 91                 	<unknown>
100007f30: 08 09 20 d9                 	<unknown>
100007f34: 09 00 80 52                 	mov	w9, #0
100007f38: 09 01 00 b9                 	str	w9, [x8]
100007f3c: 08 00 00 90                 	adrp	x8, #0
100007f40: 08 d1 3e 91                 	add	x8, x8, #4020
100007f44: ea 03 00 91                 	mov	x10, sp
100007f48: 48 01 00 f9                 	str	x8, [x10]
100007f4c: 00 00 00 90                 	adrp	x0, #0
100007f50: 00 c0 3e 91                 	add	x0, x0, #4016
100007f54: 13 00 00 94                 	bl	#76 <_printf+0x100007fa0>

If we will check the unknown instructions with llvm machine code we will see the new instructions for Memory tagging

echo "0xE8 0x13 0xC8 0x9A" | llvm-mc --disassemble -triple=aarch64 --mattr=+mte -show-encoding
	.text
	irg	x8, sp, x8              // encoding: [0xe8,0x13,0xc8,0x9a]

echo "0x08 0x01 0x81 0x91" | llvm-mc --disassemble -triple=aarch64 --mattr=+mte -show-encoding
	.text
        addg	x8, x8, #16, #0           // encoding: [0x08,0x01,0x81,0x91]

echo "0x08 0x09 0x20 0xd9" | llvm-mc --disassemble -triple=aarch64 --mattr=+mte -show-encoding
	.text
	stg	x8, [x8]                // encoding: [0x08,0x09,0x20,0xd9]

References

https://llvm.org/devmtg/2018-10/slides/Serebryany-Stepanov-Tsyrklevich-Memory-Tagging-Slides-LLVM-2018.pdf
https://developer.arm.com/docs/ddi0596/latest/base-instructions-alphabetic-order
http://www.keil.com/support/man/docs/armclang_ref/armclang_ref_lnk1549304794624.htm
https://gcc.gnu.org/gcc-9/changes.html
https://github.com/llvm-mirror/llvm/blob/1338c14d0676434da021911c8824f00425d4895d/test/MC/AArch64/armv8.5a-mte.s

Building Android AOSP for board hikey960

2019-04-06T19:30:54+00:00

Building AOSP

This instruction will show you how to build AOSP for the Hikey960.

Steps to Fetch AOSP Source:

mkdir ~/aosp
repo init -u https://android.googlesource.com/platform/manifest -b master
repo sync -j`nproc`

Set up your build environment

source build/envsetup.sh

Fetch hikey960 vendor package

./device/linaro/hikey/fetch-vendor-package.sh

Target Selection for Build: hikey960

The following example uses the current AOSP from the master branch for device hikey960. You can select the build variant as follows:

user: for production builds
userdebug: for development builds
eng: for development builds with faster build times

lunch hikey960-aosp_current-userdebug

You can enable hardware AddressSanitizer (ASAN) for memory error detection.

export SANITIZE_TARGET=hwaddress

Build AOSP: Start the build process:

m -j`nproc`

Flashing all images to the device:

Ensure the Device is in Fastboot Mode: Make sure the board is powered on with Switch 1 and Switch 3 set to ON.
Connect the Device: Connect the device to your computer via USB.
Open a Terminal: On your host machine, open a terminal window.
Verify Fastboot Connection: Run the following command to check if the device is recognised:

fastboot devices

Flash the images:

./device/linaro/hikey/installer/hikey960/flash-all.sh

Switching to Normal Mode and Connecting via ADB:

Turn off the device.
Change Jumper Switch: Set Switch 1 to the ON position and ensure Switch 2 and Switch 3 are set to OFF. This will put the board into normal mode.
Wait for the Board to Start: Allow some time for the board to fully boot up.
Connect to the Device: Once the board has started, open a terminal on your computer.
Execute ADB Shell: Run the following command to connect to the device:

adb shell

Installation is Ready: You can now proceed with your setup or configuration as needed.

Introduction to Memory Management

2019-04-06T17:30:54+00:00

MMU

The Memory Management Unit (MMU) is a hardware component which maps physical frames to virtual addresses. The MMU operates on basic units of memory called pages. Page size can be different base the architecture.

Page

Page is a unit of memory sized and aligned at the page size and page frame or frame refers to page size.

Page Table

A page table is the data structure used by a virtual memory system in a operating system to store the mapping between virtual addresses and physical addresses.

Page Faults

When a process accesses a region of memory that is not mapped or the processes has insufficient permissions for the address requested the MMU will generate a page fault exception.

Page allocator

The Linux page allocator is based on a buddy allocator, implemented in mm/page_alloc.c and tracks the free pages of different orders.

Translation Lookaside Buffer (TLB)

Translation lookaside buffer (TLB) is a memory cache that is used to reduce the time taken to access a user memory location. It is a part of the chip’s memory-management unit (MMU) and stores the recent translations of virtual memory to physical memory, also can be called an address-translation cache. TLB contains the virtual address, physical address and permissions of the mapping entries, if the the address is in the TLB the MMU will look up the physical resource, if not in TLB the MMU will generate a page fault exception and interrupt the CPU.

Non-Uniform Memory Access Numa

NUMA is used on multiprocessor systems whose memory is divided into multiple memory nodes. The access time of a memory node depends on the relative locations of the accessing CPU and accessed node, which it means that CPU access to memory depends on the distance cost between them. Memory is divided to node and each node is divided to many blocks called zones and a zone contains pages.

Memory Zones

Direct memory access (DMA) is a feature of computer systems that allows certain hardware subsystems to access main system memory (random-access memory), independent of the central processing unit (CPU). Examples of hardware systems that are using DMA are disk drive controllers, graphics cards, network cards and sound cards. Linux kernel divides pages into different zones.

cat /proc/zoneinfo

ZONE_DMA - This zone contains pages that can undergo DMA

ZONE_DMA32 - Like ZOME_DMA, this zone contains pages that can undergo DMA. Unlike ZONE_DMA, these pages are accessible only by 32-bit devices. On some architectures, this zone is a larger subset of memory.

ZONE_NORMAL - This zone contains normal, regularly mapped, pages.

Linux Kernel Memory Management APIs

kmalloc()

Kmalloc function is similar with userspace malloc() with the exception of the additional flag parameters. The kmalloc() function is a simple interface for obtaining kernel memory in byte-sized chunk and guarantees that the pages are physically contiguous (and virtually contiguous).

Further similar to calloc on userspace is kzalloc where is setting the memory with zero.

Example

p = kmalloc(sizeof(struct example), GFP_KERNEL); 

gfp_t flags

The flags are broken up into three categories: action modifiers, zone modifiers, and types. Action modifiers specify how the kernel is supposed to allocate the requested memory. Zone modifiers specify from which memory zone the allocation should originate and the type flags specify the required action and zone modifiers to fulfill a particular type of transaction.

Usual usage of flags

Process context which can sleep - GFP_KERNEL

Process context which cannot sleep - GFP_ATOMIC

Interrupt handler - GFP_ATOMIC

Softirq - GFP_ATOMIC

Tasklet - GFP_ATOMIC

DMA-able memory which can sleep - (GFP_DMA | GFP_KERNEL)

DMA-able memory which cannot sleep - (GFP_DMA | GFP_ATOMIC)

The GFP_DMA flag is used to specify that the allocator must satisfy the request from ZONE_DMA. This flag is used by device drivers, which need DMA-able memory for their devices, normally, you combine this flag with the GFP_ATOMIC or GFP_KERNEL flag.

kfree

The kfree() method frees a block of memory previously allocated with kmalloc() similar with free that we can find in userland.

kfree(p);

vmalloc

The vmalloc() function is similar to kmalloc(), except it allocates memory that is only virtually contiguous and not physically contiguous. The vmalloc() function ensures only that the pages are contiguous within the virtual address space and it does this by allocating potentially noncontinuous chunks of physical memory and “fixing up” the page tables to map the memory into a contiguous chunk of the logical address space.

p = vmalloc(16 * PAGE_SIZE);

and free with

vfree(p);

Slab Allocator

The slab allocator is an abstraction layer to make easier allocation of numerous objects of a same type. The basic idea behind the slab allocator is to have caches of commonly used objects kept in an initialized state available for use by the kernel. The Linux kernel has three main different memory allocators: SLAB, SLUB, and SLOB. The slab layer acts as a generic data structure-caching layer. In this sense, the free list acts as an object cache, caching a frequently used type of object.

Slab usage

kmem_cache_create use arguments such as name, size, flags and ctor (constructor function to call when new objects are added to the cache) to create the cache

The following flags can be used for kmem_cache_create

SLAB_NO_REAP - slab layer will reap objects in the cache when memory is low

SLAB_HWCACHE_ALIGN - align each object within a slab to different cache lines

SLAB_MUST_HWCACHE_ALIGN - debugging purpose

SLAB_POISON - fill memory with 0xa5a5a5a5, useful for catching access to uninitialized memory

SLAB_RED_ZONE - insert “red zone” around the allocated memory to detect buffer overruns

SLAB_PANIC - make slab panic() if allocation fails

SLAB_CACHE_DMA - memory allocated from the slab must come from ZONE_DMA

Example of kmem_cache_create

struct kmem_cache *kmem_cache_create(const char *name, unsigned int size, unsigned int align,slab_flags_t flags, void (*ctor)(void *));
p_cache = kmem_cache_create("Example cache",sizeof(struct example),0,SLAB_HWCACHE_ALIGN,NULL_NULL);

Once a cache of objects is created, you can allocate objects from it by calling kmem_cache_alloc

void *kmem_cache_alloc (struct kmem_cache * cachep, gfp_t flags);
p = kmem_cache_alloc(p_cache,GFP_KERNEL);

Deallocate an object

void kmem_cache_free(struct kmem_cache *cachep, void *obj);
kmem_cache_free(p_cache, p);

Finally, at module unload time, we have to return the cache to the system:

void kmem_cache_destroy(struct kmem_cache *s)
kmem_cache_destroy(p_cache);

Use ksize To find the actual amount of memory allocated for an object

size_t ksize(const void *objp);

Further you can see the kernel slab cache information in real time with slabtop

References

https://www.kernel.org/doc/htmldocs/kernel-api/mm.html
https://elixir.bootlin.com/linux/v5.1-rc5/source/kernel/sched/core.c#L2837
http://makelinux.net/ldd3/chp-8-sect-2.shtml

Introduction to AArch64 Architecture

2019-01-29T17:31:51+00:00

ARM

ARM is a family of Reduced Instruction Set Computer (RISC) architectures for computer processors that has become the predominant CPU for smartphones.

ARM Architecture

Cortex-A Highest performance Optimized for rich operating systems Application profiles implement a traditional ARM architecture with multiple modes and support a virtual memory system architecture based on an MMU(Memory management Unit). Cortex-A processors provide a range of solutions for devices that make use of a rich operating system such as Linux or Android.

Cortex-R Fast response Optimized for high-performance, hard real-time applications. Real-time profiles implement a traditional ARM architecture with multiple modes and support a protected memory system architecture based on MPU(Memory protection Unit). The Cortex-R processors target high-performance real-time applications such as hard disk controllers.

Cortex-M Smallest/lowest power Optimized for discrete processing and micro-controller. Micro-controller profiles implement a programmers’ model designed for fast interrupt processing, with hardware stacking of registers and support for writing interrupt handlers in high-level languages. The processor is designed for integration into an FPGA and is ideal for use in very low power applications.

The latest ARM architecture, ARMv8, introduces 64-bit capability alongside the existing 32-bit mode.

ARMv8 has two execution modes A64 – 64-bit registers and memory accesses, new instruction set A32 (optional) – backwards compatible with ARMv7-A.

AArch64 Privilege model

There are 4 Privilege levels: EL3 – highest, EL0 – lowest

In ARMv8, the four privilege levels extend this to provide support for virtualization and security

EL0 - This is unprivileged and is used for User-land Applications.

EL1 - This is privileged and is used for running an OS Kernel like Linux.

EL2 - This has a higher level of privilege and can be used to run a hypervisor.

EL3 - This is the highest level of privilege and is used to control (and protect) access to TrustZone.

ARMv8-A provides two security states, Secure and Non-secure. The Non-secure state is also referred to as the Normal World. This enables an Operating System (OS) to run in parallel with a trusted OS on the same hardware, and provides protection against certain software attacks and hardware attacks. EL3 is always Secure, EL2 is always Non-Secure and EL0/1 can be Secure or Non-Secure

ARM TrustZone technology enables the system to be partitioned between the Normal and Secure worlds.

TrustZone

A Trusted Execution Environment (TEE) is a secure area inside a main processor, it runs in parallel of the operating system, in an isolated environment, and it guarantees that the code and data loaded in the TEE are protected.

AArch64 Registers

General purpose Registers

Wn	32-bits	General purpose registers 0-31
Xn	64-bits	General purpose registers 0-31
WZR	32-bits	Zero register (Reads as 0, writes are ignored, a way to ignore results)
XZR	64-bits	Zero register (Reads as 0, writes are ignored, a way to ignore results)
SP	64-bits Stack pointer

X0 – X7    arguments and return value
X8 – X18   temporary registers caller-saved
X19 – X28  callee-saved registers
X29        frame pointer
X30        link register
SP         stack pointer

There are separate link registers for function calls and exceptions

X30 – Updated by branch with link instructions (BL & BLR)
      Use RET instruction to return from sub-routines

ELR_ELn – Updated on exception entry
          Use ERET instruction to return from exceptions

Each exception level has its own stack pointer SP_EL0, SP_EL1, SP_EL2 and SP_EL3

Floating point/SIMD registers

32 bit float registers

S0-S7   arguments and return value
S8-S15	Callee saved register
S16-S15 Corruptible registers
S24-S31 Corruptible registers

64 bit SIMD registers

D0-D7   arguments and return value
D8-D15	Callee saved register
D16-D15 Corruptible registers
D24-D31 Corruptible registers

128 bit SIMD registers

V0-V7   arguments and return value
V8-V15	Callee saved register
V16-V15 Corruptible registers
V24-V31 Corruptible registers

The PC (program counter) is not a general purpose register, and cannot be directly accessed by most instructions.

You can use ADR to get the the address of a PC relative offset

adr x0 label

AArch64 Exceptions

ARMv8-A exceptions interrupt the processor and change the control flow of the program.

SVC Supervisor Call attempts to access EL1 from EL0. SVC that generates a supervisor call, which are are normally used to request privileged operations or access to system resources from an operating system.
HVC Hypervisor Call attempts to access EL2
SMC Secure Monitor Call attempts to access EL3
HLT Halting software breakpoint Instruction
BRK software breakpoint instruction

Exception handling in ARMv8

In ARMv8, a new exception model has been introduced which defines the concept of exception levels. When an exception occurs, the processor branches to an exception vector table and runs the corresponding handler. In ARMv8, each exception level has its own exception vector table.

Offset from VBAR_EL1	Exception type	Exception set level
+0x000	Synchronous	Current EL with SP0
+0x080	IRQ/vIRQ	”
+0x100	FIQ/vFIQ	”
+0x180	SError/vSError	”
+0x200	Synchronous	Current EL with SPx
+0x280	IRQ/vIRQ	”
+0x300	FIQ/vFIQ	”
+0x380	SError/vSError	”
+0x400	Synchronous	Lower EL using ARM64
+0x480	IRQ/vIRQ	”
+0x500	FIQ/vFIQ	”
+0x580	SError/vSError	”
+0x600	Synchronous	Lower EL with ARM32
+0x680	IRQ/vIRQ	”
+0x700	FIQ/vFIQ	”
+0x780	SError/vSError	”

Example on Linux Kernel el1_sync

el1_sync:
 kernel_entry 1
 mrs	x1, esr_el1			// read the syndrome register
 lsr	x24, x1, #ESR_ELx_EC_SHIFT	// exception class
 cmp	x24, #ESR_ELx_EC_DABT_CUR	// data abort in EL1
 b.eq	el1_da
 cmp	x24, #ESR_ELx_EC_IABT_CUR	// instruction abort in EL1
 b.eq	el1_ia
 cmp	x24, #ESR_ELx_EC_SYS64		// configurable trap
 b.eq	el1_undef
 cmp	x24, #ESR_ELx_EC_SP_ALIGN	// stack alignment exception
 b.eq	el1_sp_pc
 cmp	x24, #ESR_ELx_EC_PC_ALIGN	// pc alignment exception
 b.eq	el1_sp_pc
 cmp	x24, #ESR_ELx_EC_UNKNOWN	// unknown exception in EL1
 b.eq	el1_undef
 cmp	x24, #ESR_ELx_EC_BREAKPT_CUR	// debug exception in EL1
 b.ge	el1_dbg
 b	el1_inv

AArch64 Instructions

Data first loaded into registers, modified, and then stored back in memory or simply discarded once it’s no longer required.

[instr] [cond] [dest]  [src] [operand]

Basic instructions

add - arithmetic instrictions

add w2,w1,#3; w2 = w1 + 3

mov - copies one register to another or load a value to a register

mov x0, x1; copies x1 into x0
mov x0, #5; move 5 to x0 register

str/ldr - store and load register

str x0, [x1]; stores X0 to memory addressed pointed by x1.  
ldr x0, [x1]; loads from memory addressed pointed by x1 to X0

stp/ldp - store and load pair of register

stp x1, x2, [sp];store x1 at sp and x2 at sp+8
ldp x29, x30, [sp]; load x29 and x30 from stack  

bl/blr - Branch link to register, jumps to a subroutine and stores the return address to x30 link register

blr x0; calls the subroutine at the address stored in x0

b/br - Branch to a register jumps to an address.

br x0;  jump to the address stored in x0

ret - it looks for the return address in the x30 register and jumps there.

ret

No load/store multiple instructions.

Conditional select

CSEL x2, x4, x5 ;cond implements x2 = if cond then x4 else x5
You can define CMOV x1, x2 with new condition  CSEL x1, x2, x1, cond	

No direct access to CPSR register but with System instruction MRS,MSR

AArch64 system configuration is controlled through system registers

TTBR0_EL1 – can be accessed from EL1, EL2 and EL3
TTBR0_EL2 – can be accessed from EL2 and EL3

Accessed using MSR and MRS instructions

MSR - Move System register to general-purpose register

MRS - Move general-purpose register to System register

Example

MRS x0, TTBR0_EL1 ; Move TTBR0_EL1 into x0
MSR TTBR0_EL1, x0 ; Move x0 into TTBR0_EL1

You can find more instructions on the following PDF

ARMv8 A64 Quick Reference

AArch64 MMU

The AArch64 MMU is used to convert from virtual address to physical address and setting the memory attributes, such as access permissions that include read and write permissions for different privilege levels, memory type, and cache policies.

AArch64 Linux uses either 3 levels or 4 levels of translation table with the 4KB page configuration, allowing 39-bit (512GB) or 48-bit (256TB) virtual addresses, respectively, for both user and kernel. With 64KB pages, only 2 levels of translation tables, allowing 42-bit (4TB) virtual address, are used but the memory layout is the same. User addresses have bits 63:48 set to 0 while the kernel addresses have the same bits set to 1. Upper 8 bits of the address can be configured for Tagged Pointers.

Further separate TTBR register for user and kernel TTBRx selection is given by bit 63 of the virtual address.

Translation Lookaside Buffer (TLB)

A translation lookaside buffer (TLB) is a memory cache that is used to reduce the time taken to access a user memory location. It is a part of the chip’s memory-management unit (MMU). The TLB stores the recent translations of virtual memory to physical memory and can be called an address-translation cache.

DoH/DoT DNS Server

2019-01-29T17:30:54+00:00

How to create your own DoH/DoT DNS Server running on Docker.

Download and install Docker

sudo apt-get update
for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  	$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin docker-compose

Create Let’s Encrypt Certificate

sudo apt-get update
sudo apt-get -y install certbot
sudo certbot certonly --no-eff-emai --agree-tos --email <your email> --standalone --non-interactive --domain <your domain>
cp /etc/letsencrypt/live/<your domain>/privkey.pem .
cp /etc/letsencrypt/live/<your domain>/fullchain.pem .

Unbound Configuration

vim unbound.conf

Inside, paste the following script:

server:
        username: "unbound"
        chroot: "/etc/unbound"
        directory: /etc/unbound/
        pidfile: "/var/run/unbound.pid"
        logfile: "/var/log/unbound.log"
        log-local-actions: yes
        log-queries: yes
        log-replies: yes
        log-servfail: yes
        verbosity: 1
        log-queries: yes
        num-threads: 4
        so-rcvbuf: 2m
        verbosity: 2
        interface: 0.0.0.0@443
        do-ip4: yes
        do-tcp: yes
        do-ip6: yes
        access-control: 0.0.0.0/0 allow
        tls-service-key: "/etc/unbound/privkey.pem"
        tls-service-pem: "/etc/unbound/fullchain.pem"
        minimal-responses: yes
        cache-min-ttl: 0
        cache-max-ttl: 86400
        hide-identity: yes
        hide-version: yes
        hide-trustanchor: yes
        minimal-responses: yes
        prefetch: yes
        prefetch-key: yes
        qname-minimisation: yes
        incoming-num-tcp: 4096
        ratelimit: 1000
        num-queries-per-thread: 4096
        rrset-roundrobin: yes
        use-caps-for-id: yes
        aggressive-nsec: yes
        edns-buffer-size: 1472
        harden-glue: yes
        harden-short-bufsize: yes
        harden-large-queries: yes
        harden-algo-downgrade: yes
        harden-dnssec-stripped: yes
        harden-below-nxdomain: yes
        harden-referral-path: yes
        do-not-query-localhost: no
        statistics-cumulative: yes
        extended-statistics: yes
        val-clean-additional: yes
        ip-dscp: 18
        private-address: 10.0.0.0/8
        private-address: 172.16.0.0/12
        private-address: 192.168.0.0/16
        private-address: fd00::/8
        private-address: 169.254.0.0/16
        private-address: fe80::/10
        private-address: 127.0.0.0/8
        private-address: ::1/128
        private-address: ::ffff:0:0/96
								
forward-zone:
        name: "."
        forward-tls-upstream: yes
        forward-no-cache: no
        forward-addr: 8.8.8.8@853
        forward-addr: 8.8.4.4@853
        forward-addr: 1.1.1.1@853
        forward-addr: 1.0.0.1@853
        forward-addr: 9.9.9.9@853
remote-control:
        control-enable: no

CMD Script

vim unbound.sh

#!/bin/bash
/usr/sbin/unbound -d -c /etc/unbound/unbound.conf

Dockerfile

vim Dockerfile

Inside, paste the following script:

FROM ubuntu:24.04

ENV DEBIAN_FRONTEND=noninteractive
ENV UNBOUND_VERSION=1.22.0
ENV UNBOUND_SHA256=c5dd1bdef5d5685b2cedb749158dd152c52d44f65529a34ac15cd88d4b1b3d43

SHELL ["/bin/bash", "-o", "pipefail", "-c"]

RUN set -x \
	&& apt update \
	&& apt -y upgrade \
	&& apt -y install build-essential net-tools libnghttp2-dev libssl-dev libexpat-dev libsodium-dev libevent-dev openssl wget knot-dnsutils \
	&& groupadd -g 88 unbound \
	&& useradd -c "Unbound DNS resolver" -d /var/lib/unbound -u 88 -g unbound -s /bin/false unbound \
	&& wget http://www.unbound.net/downloads/unbound-${UNBOUND_VERSION}.tar.gz \
	&& echo "${UNBOUND_SHA256}  unbound-${UNBOUND_VERSION}".tar.gz | sha256sum --check \
	&& tar -xvf unbound* \
	&& cd unbound-1.*/ \
	&& ./configure --with-libnghttp2 --with-libevent --enable-dnscrypt --prefix=/usr --sysconfdir=/etc --disable-static --with-pidfile=/run/unbound.pid \
	&& make \
	&& make install \
	&& mv -v /usr/sbin/unbound-host /usr/bin/ \
	&& unbound-anchor /etc/unbound/root.key  ; true\
	&& unbound-control-setup \
	&& unbound-checkconf \
	&& wget https://www.internic.net/domain/named.root -qO- | tee /etc/unbound/root.hints \
	&& rm -rf /unbound-${UNBOUND_VERSION}.tar.gz \
	&& rm -rf unbound-* \
	&& rm -rf /var/lib/apt/lists/* \
	&& apt-get clean

COPY ./unbound.conf /etc/unbound/unbound.conf
COPY ./privkey.pem /etc/unbound/privkey.pem
COPY ./fullchain.pem /etc/unbound/fullchain.pem
COPY ./unbound.sh /unbound.sh

WORKDIR /etc/unbound/

EXPOSE 443
HEALTHCHECK CMD netstat -an | grep 443 > /dev/null; if [ 0 != $? ]; then exit 1; fi;

CMD ["/unbound.sh"]

Build Image

docker build -t unbound-tls:1.22.0 .

Docker Compose

vim docker-compose.yml

Inside, paste the following script:

services:
  unbound-tls:
    image: unbound-tls:1.22.0
    container_name: unbound-tls
    restart: always
    ports:
      - "443:443"

Run image

docker compose up -d

Android Emulator Kernel Debugging

2019-01-28T17:40:51+00:00

Android Emulator Kernel Debugging

Download and compile kernel

git clone https://android.googlesource.com/kernel/goldfish
cd goldfish
git checkout android-goldfish-4.4-dev
wget https://raw.githubusercontent.com/ChAndreas/linux-config/master/config-goldfish-android
mv config-goldfish-android .config
make oldconfig
make -j64

Create emulator image

avdmanager create avd -n test-emulator -k "system-images;android-28;google_apis;x86_64"		

Start the emulator with our compiled kernel

emulator \
	-verbose \
	-show-kernel \ 
	-debug init \ 
	-avd test-emulator \
	-kernel ~/goldfish/arch/x86_64/boot/bzImage \ 
	-qemu \
	-s 

Add the following line to .gdbinit

vim ~/.gdbinit
add-auto-load-safe-path <path>/goldfish/scripts/gdb/vmlinux-gdb.py

Run gdb and attach to emulator.

gdb -q ./vmlinux
(gdb) target remote :1234

Examples of using the Linux-provided gdb helpers

Load symbols for all modules and vmlinux.

lx-symbols

Display kernel log.

lx-dmesg

List of loaded Modules.

lx-lsmod

Access current task.

p $lx_current().pid
p $lx_current().comm
p $lx_current().cred

Current kernel processes.

(gdb) lx-ps
0xffffffff822179c0 <init_task> 0 swapper/0
0xffff88003d5a0000 1 init
0xffff88003d5a1400 2 kthreadd
0xffff88003d5a2800 3 ksoftirqd/0
0xffff88003d5a5000 5 kworker/0:0H
0xffff88003d600000 7 rcu_preempt
0xffff88003d601400 8 rcu_sched
0xffff88003d602800 9 rcu_bh
0xffff88003d603c00 10 migration/0
0xffff88003d649400 11 migration/1
0xffff88003d64a800 12 ksoftirqd/1

Container_of macro is used to obtain the container structure address of given member.

(gdb) p *(struct task_struct *)0xffff880041198000
(gdb) p $container_of(init_task.tasks.next, "struct task_struct", "tasks")

Print task for specific pid.

p/x $lx_task_by_pid(1)

Print thread_info structure for the task.

p/x $lx_thread_info($lx_task_by_pid(1))

Introduction to Linux Kernel Debugging

2019-01-28T17:31:51+00:00

Kernel Debugging

Kernel debugging is used to identify kernel bugs.

Debugging by Printing

The printk is similar with printf on C standard library and can be called from anywhere in the kernel at any time, from interrupt or process context. The printk is not used on boot process, if you want to use similar debug on boot mode you can use early_printk() unless you are writing to console you can always use printk().

The printk is having the following capabilities to specify the loglevel

Loglevels that can be used on printk are below

KERN_EMERG	0	emergency condition.System maybe hang.
KERN_ALERT	1	problem that requires immediate attention
KERN_CRIT	2	critical condition
KERN_ERR	3	error
KERN_WARNING	4	warning
KERN_NOTICE	5	normal
KERN_INFO	6	informational message
KERN_DEBUG	7	debugging message
KERN_DEFAULT	d	the default kernel loglevel
KERN_CONT		continued line of printout

Example

printk(KERN_ERR "Error, return code: %d\n",ret);

pr_* macros

The pr_* macros (with exception of pr_debug) are simple shorthand definitions in include/linux/printk.h for their respective printk call and should probably be used in newer drivers. pr_devel and pr_debug are replaced with printk(KERN_DEBUG ... if the kernel was compiled with DEBUG, otherwise replaced with an empty statement. For drivers the pr_debug should not be used anymore (use dev_dbg instead).

pr_emerg
pr_alert
pr_crit
pr_err
pr_warning
pr_notice
pr_info
pr_debug,pr_devel if DEBUG is defined.
pr_cont

Example

pr_err("Error,return code: %d\n", ret);

Note that if you don’t specifying the loglevel the default is KERN_WARNING.

To determine your current console_loglevel

cat /proc/sys/kernel/printk
4	4	1	7
current	default minimum boot-time-default

To change console_loglevel

echo 8 > /proc/sys/kernel/printk

if is set to 8, all messages, including debugging ones, are displayed

Another way to change the console log level is to use dmesg with the -n parameter

 dmesg -n 7

To debug an oops and you don’t know where this happened you can use the following line

printk(KERN_ALERT "DEBUG: Passed %s %d \n",__FUNCTION__,__LINE__);

Log Buffer

Kernel messages are stored in a circular buffer of size LOG_BUF_LEN which is configurable on compile time. The default size is 16KB.

Enable Kernel debugging Option

The following options should be enabled on kernel config

CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_INFO=y

Bugs on code

There are two functions that you can use on the kernel to show an oops message.

BUG();
BUG_ON();

Further a more critical error is signaled via panic() which is printing the error and halt the system.

Example:

panic("%s: failed to map registers\n", __func__);

Sometimes we need stack trace and dump_stack() can be used which is dumps the contents of the registers and function back trace.

Example:

dump_stack();

Decoding an oops message

addr2line translates addresses into file names and line numbers. Given an address in an executable it uses the debugging information to figure out which file name and line number are associated with it.

Examples:

addr2line -f -e vmlinux 0xffffffff85037434

with gdb

gdb -q vmlinux
list *(0xffffffff85037434)

or with objdump to determine the offending line

cat /proc/modules

get the value of module and use it on objdump

objdump -dS --adjust-vma=0xffffffff85037434 vmlinux

Kernel Debugging with gdb

GDB can be used to debug the kernel. As you can see on the below command we are running gdb on uncompressed kernel image vmlinux. The kcore parameter is acting as a core file to let gdb peek into the memory

gdb -q vmlinux /proc/kcore

KGDB patch enables gdb to debug kernel remotely with serial cable.

How to enable KGDB on your kernel

CONFIG_FRAME_POINTER=y
CONFIG_KGDB=y
CONFIG_KGDB_SERIAL_CONSOLE=y
CONFIG_KGDB_KDB=y 
CONFIG_KDB_KEYBOARD=y

Example kernel Debugging with GDB and QEMU

First step is to download and compile the buildroot with the following config file.

Download buildroot

cd /root/
wget https://buildroot.uclibc.org/downloads/buildroot-2018.02.10.tar.gz
tar xvf buildroot-2018.02.10.tar.gz
cd buildroot-2018.02.10
make menuconfig

Follow the below configuration

Target options
  Target Architecture - Aarch64 (little endian)
  Target Architecture Variant (cortex-A57)  --->
Toolchain  --->
  Toolchain type (Buildroot toolchain)  --->
  Toolchain type (External toolchain)  --->
  Toolchain (Linaro AArch64 2017.11)  ---> 
    *** Host GDB Options ***
  [*] Build cross gdb for the host                                     
  [*]   TUI support
  [*]   Python support
  [*]   Simulator support
  GDB debugger Version (gdb 8.0.x)  --->  
System configuration  ---> 
[*] Enable root login with password
    ( ) Root password ⇐= set your password using this option
[*] Run a getty (login prompt) after boot --->
    TTY port - ttyAMA0
Target packages  ---> 
  [*]   Show packages that are also provided by busybox
Networking applications
    [*] dhcpcd
    [*] iproute2
    [*] openssh
Filesystem images
    [*] ext2/3/4 root filesystem
         ext2/3/4 variant (ext4)  --->
    (120M) exact size
    [*] tar the root filesystem

Build it

make -j4

Then we will need to install the aarch64 toolchain to compile the kernel

apt install gcc-aarch64-linux-gnu

Compile the Kernel with specific config

cd /root/
git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
cd linux/
wget https://raw.githubusercontent.com/ChAndreas/linux-config/master/config-aarch64
mv config-aarch64 .config
ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make oldconfig
ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make -j40

Next step is to install QEMU for the architecture that we have compiled our kernel and buildroot.

sudo apt install qemu-system-arm

Boot the QEMU with following configuration

qemu-system-aarch64 \
      -machine virt \
      -cpu cortex-a57 \
      -nographic -smp 2 \
      -hda ~/buildroot-2018.02.10/output/images/rootfs.ext4 \
      -kernel ~/linux/arch/arm64/boot/Image \
      -append "console=ttyAMA0 root=/dev/vda oops=panic panic_on_warn=1 panic=-1 ftrace_dump_on_oops=orig_cpu debug earlyprintk=serial slub_debug=UZ" \
      -m 2G \
      -net user,hostfwd=tcp::10023-:22 -net nic \
      -pidfile vm.pid -s \
       2>&1 | tee vm.log

Run the gdb and verify that everything is configured correctly by checking symbols

gdb-multiarch vmlinux
(gdb) add-auto-load-safe-path ~/linux/scripts/gdb/vmlinux-gdb.py
(gdb) info address vmalloc_user

Now on QEMU type the follow to compare with the address from the gdb

grep 'T vmalloc_user' /proc/kallsyms

The addresses should match exactly

Attach to the booted guest:

(gdb) target remote :1234

To see the source context

(gdb) info source
(gdb) list

We can see the backtrace with the following command

(gdb) bt full

Use c to continue execution, and Control-C to halt execution. Use n to step over, s to single-step, and finish to finish the current stack frame. Remember that we’re running a build with optimizations, so sometimes variables will be optimized out or the source line may change unexpectedly.

(gdb) b printk
(gdb) c

Compile Linux Kernel (Ubuntu-Debian)

2018-09-20T16:40:51+00:00

Download neccessary packages

sudo apt-get install build-essential libncurses-dev bison flex libssl-dev libelf-dev bc ccache gcc-8-plugin-dev

Download Kernel Config hardened check, to verify that we will use the best practice kernel config options for security.

git clone https://github.com/a13xp0p0v/kconfig-hardened-check.git

Download Kernel

wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.3.13.tar.xz
wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.3.13.tar.sign

Extract tar.xz kernel

unxz linux-5.3.13.tar.xz

Download gpg key of Greg Kroah-Hartman and Linus Torvalds Linux Maintainers.

gpg --locate-keys torvalds@kernel.org gregkh@kernel.org

Verify kernel tarball with gpg.

gpg --verify linux-5.3.13.tar.sign

If everything is OK and you didn’t get “BAD signature” output from the “gpg –verify” extract the tarball.

tar xvf linux-5.3.13.tar

Enter the kernel folder.

cd linux-5.3.13

Copy running kernel config.

cp -v /boot/config-$(uname -r) .config
make oldconfig

Check kernel config with kconfig-hardened-check for security.

python3 ../kconfig-hardened-check/kconfig-hardened-check.py -c .config

Make the nessesary changes that you found with kconfig-hardened-check.

make menuconfig

Build the kernel

make -j$(nproc)

Install Kernel Modules

make INSTALL_MOD_STRIP=1 modules_install -j$(nproc)

Install Kernel

make install -j$(nproc)

Update initramfs

update-initramfs -u -k all

Reboot your system

reboot

ARM Pointer Authentication

2018-09-20T16:35:51+00:00

The ARMv8.3 pointer authentication extension adds functionality to detect modification of pointer values, mitigating certain classes of attack such as ROP/JOP attacks. In essence, it attaches a cryptographic signature to pointer values. Those signatures can be verified before a pointer is used. An attacker with lack of the key used to create the signatures, cannot create valid pointers for use in an exploit.

When CONFIG_ARM64_POINTER_AUTHENTICATION is selected, and relevant HW support is present, the kernel will assign a random APIAKey value to each process at exec*() time. The key is assigned at exec() time, not at process creation. So threads share a key, and parent and child will share a key after fork() until one of them calls exec().

New instructions are added which can be used to:

Insert a PAC into a pointer
Strip a PAC from a pointer
Authenticate strip a PAC from a pointer

PAC instruction sign pointers become not usable pointer (Pointer + PAC)

AUT instruction authenticate PAC if PAC match the result of the original pointer if not the result will be an invalid pointer (fault).

XPAC instruction stip PAC remove authentication and restored to the original pointer

The new PAC instruction can be used to calculate the authentication code and stored within pointer value. The value containing the authentication code cannot be dereferenced directly, since, without the sign-extension bits, it is no longer recognized as a valid address.

Regaining a usable pointer requires using the AUT instruction, which will recalculate the authentication code and compare it to what is found in the authenticated pointer value. If the two match, the authentication code will be removed otherwise, the pointer will be modified to ensure a fault should it be dereferenced. Thus, any attempt to use a pointer that lacks a proper authentication code will lead to a crash.

GCC 7 compiler include basic support for pointer authentication in the form of the -msign-return-address option.

Example without arm pointer authentication

0000000000000724 <main>:
a9bf7bfd	stp	x29, x30, [sp, #-16]!
910003fd	mov	x29, sp
72c:	90000000	adrp	x0, 0 <_init-0x598>
911fa000	add	x0, x0, #0x7e8
97ffffb7	bl	610 <printf@plt>
d503201f	nop
73c:	a8c17bfd	ldp	x29, x30, [sp], #16
d65f03c0	ret
00000000	.inst	0x00000000 ; undefined

Example with arm pointer authentication

0000000000000724 <main>:
724:	d503233f	paciasp
728:	a9bf7bfd	stp	x29, x30, [sp, #-16]!
72c:	910003fd	mov	x29, sp
730:	90000000	adrp	x0, 0 <_init-0x598>
734:	911fc000	add	x0, x0, #0x7f0
738:	97ffffb6	bl	610 <printf@plt>
73c:	d503201f	nop
740:	a8c17bfd	ldp	x29, x30, [sp], #16
744:	d50323bf	autiasp
748:	d65f03c0	ret
74c:	00000000	.inst	0x00000000 ; undefined

Control Flow Integrity

2018-09-20T16:15:51+00:00

To improve security modern systems contain many mitigation strategies that try to make it harder to exploit security vulnerabilities. One of those mitigation technologies that is recently added in Android 9 is Control Flow Integrity.

CFI is a security mechanism that disallows changes to the original control flow graph of a compiled binary(Control-flow hijacking: Arbitrary code execution), making it significantly harder to perform such attacks.

CFI restricts the control-flow of an application to valid execution traces by monitoring the program runtime and compare it with precomputed state and if an invalid state an alert is raised and terminate the app.

The CFI mechanism consists of two abstract components: the (often static) analysis component that recovers the Control-Flow Graph (CFG) (Paths defined by application’s Control-Flow Graph) of the application (at different levels of precision) and the dynamic enforcement mechanism that restricts control flows according to the generated CFG.

CFI assumes that executable code is read-only, otherwise an attacker could simply overwrite code and remove the runtime monitors.

Indirect control-flow transfers are further divided into forward-edge control-flow transfers and backward-edge control-flow transfers.

Forward-edge control-flow transfers direct code forward to a new location and are used in indirect jump and indirect call instructions, which are mapped at the source code level to, e.g., switch statements, indirect calls, or virtual calls.

The backward-edge is used to return to a location that was used in a forward-edge earlier, e.g., when returning from a function call through a return instruction.

Example CFI on Android Makefile (Implementing system CFI)

Android.mk

LOCAL_SANITIZE := cfi
# Optional features
LOCAL_SANITIZE_DIAG := cfi
LOCAL_SANITIZE_BLACKLIST := cfi_blacklist.txt

Example CFI in blueprint files

Android.bp

sanitize: {
    cfi: true,
    diag: {
        cfi: true,
    },
    blacklist: "cfi_blacklist.txt",
},

Introduction to Kubernetes Cluster

2018-05-08T17:31:51+00:00

Kubernetes

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

Installation

Distro: Ubuntu 16.04
Servers: Master
	Node Server 1
	Node Server 2

Disable the swap file on all servers

sudo swapoff -a

Remove any matching reference found in /etc/fstab

Install the Docker

apt-get update

apt-get install -y apt-transport-https ca-certificates curl software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -

add-apt-repository "deb https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(lsb_release -cs) stable"

apt-get update && apt-get install -y docker-ce=$(apt-cache madison docker-ce | grep 17.03 | head -1 | awk '{print $3}')

Install Kubernetes packages

You will install these packages on all of your machines:

kubeadm: the command to bootstrap the cluster. kubelet: the component that runs on all of the machines in your cluster and does things like starting pods and containers. kubectl: the command line util to talk to your cluster.

apt-get update && apt-get install -y apt-transport-https curl

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -

cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF	

apt-get update

apt-get install -y kubelet kubeadm kubectl kubernetes-cni

Create the cluster

We need to create the cluster by initiating the master with kubeadm. Only do this on the master node.

The Kubernetes master is responsible for maintaining the desired state for your cluster.

sudo kubeadm init --pod-network-cidr=172.16.0.0/16

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Login to the other two nodes which are the worker nodes and use the token to join them to the master node. The worker nodes in a cluster are the machines (VMs, physical servers, etc) that run your applications and cloud workflows.

kubeadm join <ip>:6443 --token <token key> --discovery-token-ca-cert-hash sha256:<key>

Install networking

Only on master execute the following

Weave NET

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
curl -SL "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')&env.IPALLOC_RANGE=172.16.0.0/16" | kubectl apply -f -

Creating Pods, Deployments and Services

Pod

A Kubernetes Pod represents a running process on your cluster. A pod is a group of one or more containers, with shared storage/network, and a specification for how to run the containers.

Example Create a pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:

  restartPolicy: Always

  volumes:
  - name: shared-data
    emptyDir: {}

  containers:

  - name: nginx-container
    image: nginx
    volumeMounts:
    - name: shared-data
    mountPath: /usr/share/nginx/html

kubectl create -f pod.yaml

Deployment

A kubernetes Deployment manages creating Pods by means of ReplicaSets.

ReplicaSet is the next-generation Replication Controller.A ReplicaSet ensures that a specified number of pod replicas are running at any given time.

create a deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

kubectl create -f deployment.yaml

Replicas can be changed either by editing the deployment or using scale.

kubectl edit deployment nginx
	
kubectl scale --replicas=3 deployment/nginx

Service

A Kubernetes Service is an abstraction which defines a logical set of Pods and a policy by which to access them. . The set of Pods targeted by a Service is determined by a Label Selector

create service.yaml file

kind: Service
apiVersion: v1
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
  - protocol: TCP
    port: 80
    targetPort: 9376

kubectl create -f service.yaml

Sources

https://kubernetes.io/docs/

Introduction to LSM Linux security Module Framework

2018-05-08T17:31:51+00:00

The Linux Security Module framework provides a mechanism for various security checks to be hooked by new kernel extensions.
The LSM are not actually loadable kernel modules, instead are selectable at build time via CONFIG_DEFAULT_SECURITY and can be overridden at boot-time via the “security=…” in case that more than LSM built in to the kernel. Linux Mandatory Access Control (MAC) or policy based access control used to enable addition checks base on a policy describing what kind of operations are allowed for which process in which context. Examples of LSM interface are SELinux, Smack, Tomoyo, and AppArmor. Without enabling a specific LSM to the kernel the Linux capabilities will be a default LSM.

To see the enabled and active security modules use the following command.

cat /sys/kernel/security/lsm

Example of Linux Security Module.

YAMA

Yama is a Linux Security Module that collects system-wide DAC security protections that are not handle by the core kernel itself. It does Discretionary Access Control of some kernel related functions, like defining if process tracing (prace) is allowed. Linux DAC permissions are classical unix checks compare the current process with UID GID versus the UID and GID of the file being accessed with the checks of which modes are been set “read/write/execute”.

To enable YAMA use CONFIG_SECURITY_YAMA=y on kernel config.

kernel.yama.ptrace_scope this parameter helps system administrators to select what processes can be debugged with ptrace.

To determine the active value of yama ptrace scooe we can use sysctl or check pseudo file system /proc and find the related key.

sysctl kernel.yama.ptrace_scope

cat /proc/sys/kernel/yama/ptrace_scope


kernel.yama.ptrace_scope = 0: all processes can be debugged, as long as they have same uid. This is the classical way of how ptracing worked.
kernel.yama.ptrace_scope = 1: only a parent process can be debugged.
kernel.yama.ptrace_scope = 2: Only admin can use ptrace, as it required CAP_SYS_PTRACE capability.
kernel.yama.ptrace_scope = 3: No processes may be traced with ptrace. Once set, a reboot is needed to enable ptracing again.

Examples of the two most popular MAC extensions are below.

Apparmor

AppArmor is MAC style security extension for the Linux kernel. It implements a task centered policy, with task “profiles” being created and loaded from user space. Tasks on the system that do not have a profile defined for them run in an unconfined state which is equivalent to standard Linux DAC permissions.

To enable apparmor use CONFIG_SECURITY_APPARMOR=y on kernel config

Check apparmor status

apparmor_status

Selinux

Selinux is MAC style security extension built into the Linux kernel and defines the access and transition rights of each user, applications, processs and files on the system. The decision making process when you try to access an object example file the policy enforcement server in the kernel will checks an access vector cache AVC and checks if you can access the file or not.

To enable selinux use CONFIG_SECURITY_SELINUX=y on kernel config.

There are three modes on Selinux enforcing which will check and deny or grant an access, permissive where AVC will check and log the attemp if is denied and selinux will not enforce the policy and last mode is disabled where the selinux is fully disabled.

Check Selinux status

sestatus -v

Sources

https://www.kernel.org/doc/html/v4.16/admin-guide/LSM/index.html
https://linux-audit.com/protect-ptrace-processes-kernel-yama-ptrace_scope/

Compile aarch64 Linux kernel Upstream for Raspberry Pi

2018-05-08T17:31:51+00:00

This is a guide how to compile and install 64 bit (aarch64) kernel for the Raspberry pi 3 b, 3 b+ or 4.

Install build tools

sudo apt-get install build-essential libgmp-dev libmpfr-dev libmpc-dev bc git-core binutils-aarch64-linux-gnu gcc-aarch64-linux-gnu cpp-aarch64-linux-gnu g++-aarch64-linux-gnu libncurses-dev

Clone Linux kernel sources

git clone https://github.com/raspberrypi/linux.git rpi3-kernel
cd rpi3-kernel

Create output folder

mkdir ../out

Compile kernel

make O=../out/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcmrpi3_defconfig
make -j4 O=../out/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
mkdir ../out/out-modules/
make O=../out/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- INSTALL_MOD_PATH=../out/out-modules/ modules_install

Install (plug the sdcard and replace sdX with your device)

mount /dev/sdX1 /media/
cd ../out/
rm -rf /media/overlays
cp -R arch/arm64/boot/dts/overlays /media/
cp arch/arm64/boot/Image /media/kernel8.img
cp arch/arm64/boot/dts/broadcom/bcm2710-rpi-3-b.dtb /media/bcm2710-rpi-3-b.dtb
echo "arm_64bit=1" >> /media/config.txt
echo "kernel=kernel8.img" >> /media/config.txt
umount /media
mount /dev/sdX2 /media/
cd out-modules/lib/modules/
cp -R 4.14.* /media/lib/modules/
umount /media

Linux Namespaces

2018-01-14T17:31:51+00:00

Namespaces are a feature on the Linux kernel that isolate and virtualize system resources of a collection of processes. Examples of softwares using namespaces are Docker, LXC and Mesos.

There are two essential set of namespaces, label based namespaces and mapping based namespaces.

Label based namespace are used to attach resources to a namespace, for example when you want to add a network card on a namespace you will not see it on the host and it will reappear when your remove it from the namespace. The first label based namespace was network namespace.

Mapping namespaces are used to map resources from parent process to namespace itself, for example pid namespace takes the process pid and projects it into the namespace with different pid. Very useful when you want to run a system container and you want to run a service that required init.

To enable namespaces support on your kernel you must enable if not already the below configs on your kernel .config file:

CONFIG_NAMESPACES
CONFIG_UTS_NS
CONFIG_IPC_NS
CONFIG_USER_NS
CONFIG_PID_NS
CONFIG_NET_NS

Linux supports 7 Namespaces:

Mount - isolate filesystem mount points
UTS - isolate hostname and domain name
IPC - isolate interprocess communication (IPC) resources
PID - isolate the PID number space
Network - isolate network interfaces
User - isolate UID/GID number spaces
Cgroup - isolate cgroup root directory

The namespaces API includes the following system calls:

clone - creates a new process similar to fork() but allows child process to share parts of its execution.
setns - allows the calling process to join an existing namespace and allow any type of namespace to be joined.
unshare - moves the calling process to a new namespace and allows a process to disassociate parts of its execution that currently being shared with other process.

Mount:

Used to control mount points. Process can have their own rootfs, private or share mount.

Mount namespaces serve a variety of purposes. For example, they can be used to provide per-user views of the filesystem. Other uses include mounting a /proc filesystem for a new PID namespace without causing side effects for other process and chroot()-style isolation of a process to a portion of the single directory hierarchy. In some use cases, mount namespaces are combined with bind mounts.

Example

#define _GNU_SOURCE
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <sched.h>
#include <signal.h>
#include <unistd.h>
#include <sys/mount.h>

#define STACK_SIZE (1024 * 1024)

static char child_stack[STACK_SIZE]; /* Stack size for cloned child */

int child_main(void* arg)
{
    if (mount("proc", "/proc", "proc", MS_BIND|MS_REC|MS_SHARED, NULL) == -1)
        perror("mount");
    char *child[] = {
                    "/bin/bash",
                    "-c",
                    "cat /proc/$$/mountinfo | sed 's/ - .*//'",
                    NULL
                    };
    execvp(child[0], child);
    return 1;
}

int main(int argc, char *argv[])
{
        int child_pid = clone(child_main, child_stack+STACK_SIZE,
            CLONE_NEWNS | SIGCHLD, NULL);

  waitpid(child_pid, NULL, 0);
  return 0;
}

UTS:

Used to set a hostname on the container.The main job of this namespace was to make nfs run in container.

Example

#define _GNU_SOURCE
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <sched.h>
#include <signal.h>
#include <unistd.h>

#define STACK_SIZE (1024 * 1024)

static char child_stack[STACK_SIZE]; /* Stack size for cloned child */

int child()
{
    sethostname("Container", 10);
    char *shell[] = {
                    "/bin/bash",
                    "-c",
                    "echo $HOSTNAME",
                    NULL
                    };
    execvp(shell[0], shell);
    return 0;
}

int main(int argc, char *argv[])
{
    int child_pid = clone(child, child_stack+STACK_SIZE,
            CLONE_NEWUTS | SIGCHLD, NULL);
    waitpid(child_pid, NULL, 0);
    return 0;
}

IPC:

This namespace allows to unshare IPCs to have an isolated set of System V IPC objects (sem, shm, msg) and POSIX message queues inside namespace.

#define _GNU_SOURCE
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <sched.h>
#include <signal.h>
#include <unistd.h>

#define STACK_SIZE (1024 * 1024)

static char child_stack[STACK_SIZE]; /* Stack size for cloned child */

int child_main(void* arg)
{
    char *child[] = {
                    "/bin/bash",
                    "-c",
                    "ipcs -m",
                    NULL
                    };
    execvp(child[0], child);
    return 1;
}

int main(int argc, char *argv[])
{
	int child_pid = clone(child_main, child_stack+STACK_SIZE,
            CLONE_NEWIPC | SIGCHLD, NULL);

  waitpid(child_pid, NULL, 0);
  return 0;
}

PID:

Process within the PID namespace can see see process only in the same PID namespace and the PID will start with number 1.

Example

#define _GNU_SOURCE
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <sched.h>
#include <signal.h>
#include <unistd.h>

#define STACK_SIZE (1024 * 1024)

static char child_stack[STACK_SIZE]; /* Stack size for cloned child */

int child_main(void* arg)
{
    char *child[] = {
                    "/bin/bash",
                    "-c",
                    "echo $$",
                    NULL
                    };
    execvp(child[0], child);
    return 0;
}

int main(int argc, char *argv[])
{
    int child_pid = clone(child_main, child_stack+STACK_SIZE,
			CLONE_NEWPID | SIGCHLD, NULL);
	waitpid(child_pid, NULL, 0);
	return 0;
}

Network:

Example if you run

    ip addr

you will see all the network interface of your system, but lets create a network namespace named test

    ip netns add test

and run again

    ip netns exec test ip addr

you will see only loopback interface. If you want you can add interfaces.

    ip link set enp1s0 netns test.

to remove it

    ip netns exec test ip link set enp1s0 netns 1

User:

User namespace gives enhanced privileges to a user and can emulate root user. Also used to map users and groups.If you run namespaces without the CLONE_NEWUSER flag is placed in the same user namespace as its parent process.

Example

#define _GNU_SOURCE
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <sched.h>
#include <signal.h>
#include <unistd.h>

#define STACK_SIZE (1024 * 1024)
 
static char child_stack[STACK_SIZE]; /* Stack size for cloned child */
 
int child_main(void* arg)
{ 
    char *child[] = {
                    "/bin/bash",
	            "-c",
		    "id",
		    NULL,
                    };
    execvp(child[0], child);
    return 0;
}
 
int main(int argc, char *argv[])
{
    int child_pid = clone(child_main, child_stack+STACK_SIZE,
             CLONE_NEWUSER | SIGCHLD, NULL);
    waitpid(child_pid, NULL, 0);
    return 0;
}

Cgroups:

A process can call unshare() using the CLONE_NEWCGROUP flag to enter a new cgroup namespace. Once it does that, it will no longer see the global cgroup hierarchy.

#define _GNU_SOURCE
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <sched.h>
#include <signal.h>
#include <unistd.h>

#ifndef CLONE_NEWCGROUP
#define CLONE_NEWCGROUP         0x02000000
#endif

#define STACK_SIZE (1024 * 1024)

static char child_stack[STACK_SIZE]; /* Stack size for cloned child */

int child_main(void* arg)
{
    char *child[] = {
                    "/bin/bash",
	            "-c",
	            "cat /proc/self/cgroup",
	            NULL,
                    };
    execvp(child[0], child);
    return 0;
}

int main(int argc, char *argv[])
{
  int child_pid = clone(child_main, child_stack+STACK_SIZE,
            CLONE_NEWCGROUP | SIGCHLD, NULL);
  waitpid(child_pid, NULL, 0);
  return 0;
}

unshare

The unshare command allows you to run a program with some namespaces ‘unshared’ from its parent.

Example

sudo unshare --map-root-user --user bash -c id

setns

joining an existing namespace

Example

#define _GNU_SOURCE
#include <fcntl.h>
#include <sched.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>

int main(int argc, char *argv[])
{
    int fd;
    if (argc < 3) {
        fprintf(stderr, "%s /proc/PID/ns/FILE cmd arguments\n", argv[0]);
        exit(EXIT_FAILURE);
    }

    fd = open(argv[1], O_RDONLY);
    if (fd == -1)
        perror("open");

    if (setns(fd, 0) == -1)
        perror("setns");

    execvp(argv[2], &argv[2]);
         perror("execvp");
}

Sources

https://linux.die.net/man/2/setns
https://en.wikipedia.org/wiki/Linux_namespaces
http://man7.org/linux/man-pages/man7/namespaces.7.html
https://lwn.net/Articles/531114/

Linux Capabilities

2018-01-13T18:31:51+00:00

Linux Kernel distinguishes its processes with the following two categories:

1. Privileged Processes: These processes allow the user to bypass all Kernel permission checks.
2. Unprivileged Processes: These processes are subject to full permission checks, such as the effective UID, GID, and supplementary group list.

The goal of capabilities is divide the power of superuser into pieces, such that if a program that has one or more capabilities is compromised, its power to do damage to the system would be less than the same program running with root privilege.

Linux’s thread/process privilege checking is based on capabilities.

List of some of the capabilities that are enabled on Linux:

CAP_AUDIT_CONTROL - Allows enable and disable kernel auditing.
CAP_AUDIT_READ - Allows reading the audit log via a multicast netlink socket.
CAP_AUDIT_WRITE - Allows to write records to kernel auditing log.
CAP_BLOCK_SUSPEND - Allows the ability to block system suspend.
CAP_CHOWN - Allows to make arbitrary changes to file UIDs and GIDs
CAP_DAC_OVERRIDE - Allows to bypass file read, write, and execute permission checks.
CAP_DAC_READ_SEARCH - Allows to bypass file read permission checks and directory read and execute permission checks.
CAP_NET_ADMIN - Allows various  network-related  operations.
CAP_NET_RAW - Allows to use of RAW and PACKET sockets.
CAP_SETUID - Allows arbitrary manipulations of process UIDs.
CAP_SETGID - Allows arbitrary manipulations of process groups id.
CAP_SETPCAP - Allows to grant or remove any capability in the caller's permitted capability set to or from any other process.
CAP_MKNOD - Allows to create filesystem node 
CAP_SYS_MODULE - Allows to load and unload modules
CAP_SYS_NICE - Allows change process nice value
CAP_SYS_PTRACE - Allows to trace process
CAP_SYS_TIME - Allows to set system clock
CAP_SYS_CHROOT - Allows to run chroot.
CAP_SYS_ADMIN - Allows a range of system administration operation (dangerous capability)
CAP_MAC_OVERRIDE - Allows to change mac address.
CAP_NET_BIND_SERVICE - Allows to bind privileged ports(less than 1024)
CAP_NET_BROADCAST - Allows socket broadcast and listen to multicast.
CAP_SETFCAP - Allows to set file capabilities.
CAP_SYS_BOOT - Allows to run reboot.
CAP_SYS_PACCT - Allows to enable or disable process accounting
CAP_SYS_RAWIO - Allows I/O port operation.
CAP_KILL - Allows to send kill signals.

There are 3 modes for Capabilities:

e: Effective - This indicates that the capability is "activated."
p: Permitted - This indicates that the capability can be used.
i: Inherited - This indicates that the capability is inherited by child elements/subprocesses and defines which capabilities stay permitted across an exec().

Getting/Setting capabilities from userland:

getcap - get capabilities
setcap - set capabilities

Example to allow user run wireshark dumpcap utility without root privileges:

Run getcap you will see that no capabilities are enabled for dumpcap

    getcap /usr/bin/dumpcap

Run the below command to allow raw socket and network administrator privileges for dumpcap

    sudo setcap cap_net_raw,cap_net_admin+eip  /usr/bin/dumpcap

Run again the getcap commnad and you will get the output that dumpcap has the new capabilities.

    /usr/bin/dumpcap = cap_net_admin,cap_net_raw+eip

Example check capabilities on c code.

#define  _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <linux/capability.h>
#include <sys/capability.h>
#include <errno.h>

int cap_check(cap_value_t cap_flag)
{
    cap_t cap;
    cap = cap_get_proc();
    
    if (cap == NULL)
        return -1; 
    
    if (cap_set_flag(cap, CAP_EFFECTIVE, 1, &cap_flag, CAP_SET) == -1 )
    {
        perror("not enabled");
        cap_free(cap);
        return -1;
    }
    if (cap_set_proc(cap) != 0 )
    {
        perror("cannot set");
        cap_free(cap);
        return -1;
    }
    return 0;
}

int main(int argc, char *argv[])
{
    FILE *fd;
    int c;

    if (cap_check(CAP_DAC_READ_SEARCH) == -1)
    {   perror("cap_set_flag CAP_DAC_READ_SEARCH");
        return -1;
    }
    
    if (cap_check(CAP_DAC_OVERRIDE) == -1)
    {
        perror("cap_set_flag CAP_DAC_OVERRIDE");
        return -1;
    }
    if (cap_check(CAP_SYS_ADMIN) == -1)
    {
        perror("cap_set_flag CAP_SYS_ADMIN");
        return -1;
    }

    fd = fopen ("/etc/shadow", "r");
    if (fd == NULL)
    {
        printf("Open /etc/shadow failed\n");
        return -1;
    } else {
        while(1) {
            c = fgetc(fd);
            if( feof(fd) ) { 
                break ;
            }
            printf("%c", c);
        }
        fclose(fd);
    }
    return 0;
}

sources

https://www.kernel.org/pub/linux/libs/security/linux-privs/kernel-2.2/capfaq-0.2.txt
http://man7.org/linux/man-pages/man7/capabilities.7.html
http://man7.org/linux/man-pages/man3/cap_get_proc.3.html

Introduction to Control Groups (cgroups)

2017-12-21T17:15:51+00:00

Cgroups is a Linux kernel feature to limit accounts and isolate the resources usage of process.

The cgroup model is one or more separate, unconnected trees of tasks and many different hierarchies of cgroups can exist simultaneously on a system. A single hierarchy can have one or more subsystems attached to it, but any single subsystem (such as cpu) cannot be attached to more than one hierarchy if one of those hierarchies has a different subsystem attached to it already.

Multiple separate hierarchies of cgroups are necessary because each hierarchy is attached to one or more subsystems. A subsystem (resource controller) represents a single resource such as CPU time or memory.

Each time a new hierarchy is created on the systems, all tasks on the system are initially members of the default cgroup of that hierarchy, which is known as the root cgroup.

Further a child task automatically inherits the cgroup membership of its parent but can be moved to different cgroups as needed.

You can find enabled control groups on your computer via proc filesystem:

    cat /proc/cgroups or
      
    ls -l /sys/fs/cgroup/

Linux kernel provides support for following control group subsystems:

cpuset -  provide a mechanism for assigning a set of CPUs and Memory Nodes to a set of tasks.
cpu - uses the scheduler to provide cgroup tasks access to the processor resources.
cpuacct - account the CPU usage of these groups of tasks.
blkio - sets limits on input/output access to and from block devices such as physical drives (disk, solid state, or USB).
memory - isolates the memory behaviour of a group of tasks from the rest of the system.
devices - allows or denies access to devices by tasks in a cgroup. 
freezer - start or stop sets of tasks or a task in order to schedule the resources of a machine.
net_cls - this subsystem tags network packets with a class identifier (classid) that allows the Linux traffic controller (tc) to identify packets originating from a particular cgroup task. 
net_prio - provides a dynamically way to set the priority  of network traffic per network interface.
perf_event - provides access to perf events  to a group.
hugetlb - allows to limit the HugeTLB (hugepagesize) usage per control group and enforces the controller limit during page fault.
pids - Used to allow a cgroup hierarchy to stop any new tasks from being fork()'d or clone()'d after a certain limit is reached.
systemd - create a cgroup for each service when starting it.

Example

Systemd with Cgroups

Through Systemd, we can limit resources for our test service

Create and compile a test c program that is using 100% of a single core and put it on /root folder

#include <stdio.h>
int main()
{    
    for (;;);
    return 0;
}

Create the following systemd service with limited cpu to 20% percentage and share to 200 (default is 1024 for each core) and memory limit to 50M

/etc/systemd/system/test.service

[Unit]
Description=Stress CPU

[Service]
ExecStart=/root/test
ExecStop=/bin/kill -WINCH ${MAINPID}
MemoryAccounting=true 
CPUShares=200
CPUQuota=20%
MemoryLimit=50M

[Install]
WantedBy=multi-user.target

Start service:

systemctl start test.service

Run systemd-cgtop to see the usage of our test service.

The result should be something like below:

Path                      Tasks   %CPU   Memory  Input/s Output/s
/system.slice/test.service  1     19.7    84.0K     -        -

To see the difference just remove from the service the CPUShares and CPUQuota to see that the CPU usage for a single core will be 100%:

Path                      Tasks   %CPU   Memory  Input/s Output/s
/system.slice/test.service  1     99.4    88.0K     -        -

To change the values when the service is running and save this:

systemctl set-property test CPUShares=500 CPUQuota=50% MemoryLimit=100M

Docker with Cgroups

We can create a container that is running our test software and we can set the container to run with ‘cpu.shares’ to assign a relative CPU share and how many cores with cpuset.cpus.

Example

docker run -d \
--name='test_with_cgroup' \
--cpuset-cpus=0 \
--cpu-shares=50 \
/bin/bash /tmp/test

Again to see the difference just remove the lines CPUShares and CPUset to see that the CPU usage for a single core will be 100%

Limiting Users Resources with cgroups

Create config file /etc/cgconfig.conf

group test {
     	cpu {
		cpu.shares="25";
     	}
     	cpuacct {
            	cpuacct.usage="0";
     	}
     	memory {
            	memory.limit_in_bytes="1G";
            	memory.memsw.limit_in_bytes="1G";
     	}
}

Create config file /etc/cgrules.conf

#<user/group>         <controller(s)>         <cgroup>
@test                 cpu,cpuacct,memory        test

Run configuration parcer to parse and load the specified cgroups configuration file

cgconfigparser -l /etc/cgconfig.conf

Run the Rule engine daemon

cgrulesengd

Or you can use systemd to start and enable it:

systemctl enable cgconfig
systemctl start cgconfig

Sources

https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt
https://www.kernel.org/doc/Documentation/cgroup-v1/cpusets.txt
https://www.kernel.org/doc/Documentation/cgroup-v1/devices.txt
https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt
https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt
https://www.kernel.org/doc/Documentation/cgroup-v1/blkio-controller.txt
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/ch01

Kernel Tracing with Ftrace

2017-12-06T17:15:51+00:00

Ftrace is a tracing utility in the the Linux kernel, designed to help out developers of the system to find what is going on inside the kernel.

Ftrace was developed by Steven Rostedt and has been included in the kernel since version 2.6.27.

It can be used for debugging or analyzing latencies and performance issues that take place outside of user-space.

Ftrace uses the debugfs file system to hold the control files as well as the files to display output.

To mount this directory, you can add to your /etc/fstab file:

tracefs       /sys/kernel/debug/tracing       tracefs defaults        0       0

Or you can mount it at run time with:

mount -t tracefs nodev /sys/kernel/debug/tracing

cd /sys/kernel/debug/tracing

Some of the key files are

ls /sys/kernel/debug/tracing

available_tracers – available tracing programs
tracing_on - 0 into this file to disable the tracer or 1 to enable it
trace - holds the output of the trace
current_tracer – display the current tracer that is configured

Current tracers that may be configured

cat /sys/kernel/debug/tracing/available_tracers

function - Function call tracer to trace all kernel functions.
function_graph - Similar to the function tracer except that the function tracer probes the functions on their entry
	     whereas the function graph tracer traces on both entry and exit of the functions. It then provides the ability to draw a graph of function calls similar to C code source.
blk -  The block tracer I/O. The tracer used by the blktrace user application.
hwlat - The Hardware Latency tracer is used to detect if the hardware produces any latency.
irqsoff - Traces the areas that disable interrupts and saves the trace with the longest max latency.
preemptoff - Similar to irqsoff but traces and records the amount of time for which preemption is disabled.
preemptirqsoff - Similar to irqsoff and preemptoff, but traces and records the largest time for which irqs and/or preemption is disabled.
wakeup - Traces and records the max latency that it takes for the highest priority task to get scheduled after it has been woken up.
wakeup_rt - Traces and records the max latency that it takes for just RT tasks (as the current "wakeup" does).
wakeup_dl - Traces and records the max latency that it takes for a SCHED_DEADLINE task to be woken (as the "wakeup" and "wakeup_rt" does).
mmiotrace - A special tracer that is used to trace binary module. It will trace all the calls that a module makes to the hardware. Everything it writes and reads from the I/O as well.
branch - This tracer can be configured when tracing likely/unlikely calls within the kernel. It will trace when a likely and unlikely branch is hit and if it was correct in its prediction of being correct.
nop - This is the "trace nothing" tracer. A way to disable other tracers and can be use with events

Example how to run ftrace with function tracer

sysctl kernel.ftrace_enabled=1
echo function > /sys/kernel/debug/tracing/current_tracer
echo 1 > /sys/kernel/debug/tracing/tracing_on
sleep 1
echo 0 > /sys/kernel/debug/tracing/tracing_on
cat /sys/kernel/debug/tracing/trace

Limiting the traces with setting up the filters

set_ftrace_filter - enabling or disabling specific functions to be traced
set_ftrace_notrace -  Any function that is added here will not be traced
set_ftrace-pid - only traces functions executed by a task with the given pid
set_graph_function - Functions listed in this file will cause the function graph tracer to only trace these functions and the functions that they call.
set_graph_notrace - Disable function graph tracing when the function is hit until it exits the function.

Function name filter is very limited show only when enter or exit the function

echo schedule > set_ftrace_filter

Appending the filter

echo '*rcu*' >> set_ftrace_filter

Clear the filter

echo > set_ftrace_filter

Example filter syscalls

echo SyS_read > set_ftrace_filter

Function stack trace with func_stack_trace option

echo schedule > set_ftrace_filter
echo 1 > /sys/kernel/debug/tracing/options/func_stack_trace
echo function > /sys/kernel/debug/tracing/current_tracer
echo 0 > /sys/kernel/debug/tracing/tracing_on
ech0 0 > /sys/kernel/debug/tracing/options/func_stack_trace

Tracing Events

The trace event directory. It holds event tracepoints (also known as static tracepoints) that have been compiled into the kernel. Tracepoints placed in code provides a hook to call a function (probe) that you can provide at runtime can be used without creating custom kernel modules to register probe functions using the event tracing infrastructure.

You can find all events on

cat /sys/kernel/debug/tracing/available_events

Example with events

echo 0 > /sys/kernel/debug/tracing/tracing_on
echo 1 > /sys/kernel/debug/tracing/events/syscalls/enable
echo 1 > /sys/kernel/debug/tracing/events/exceptions/enable
sh -c 'echo $$ > /sys/kernel/debug/tracing/set_event_pid; echo 1 > /sys/kernel/debug/tracing/tracing_on; exec echo ftrace'
cat /sys/kernel/debug/tracing/trace

set_event_pid Events only trace a task with a PID listed in this file

echo 0 > /sys/kernel/debug/tracing/tracing_on
    echo 1 > /sys/kernel/debug/tracing/events/syscalls/enable
    echo 1 > /sys/kernel/debug/tracing/events/exceptions/enable		
echo $$ > /sys/kernel/debug/tracing/set_event_pid 
echo 1 > /sys/kernel/debug/tracing/tracing_on; 
echo ftrace_events;
echo 0 > /sys/kernel/debug/tracing/tracing_on;

The new interface trace-cmd

Change tracer to function

trace-cmd start -p function

See available filters

trace-cmd list -f

Set ftrace filter

trace-cmd start -p function -l schedule

Set ftrace pid

trace-cmd record -p function -F echo ftrace

Syscall tracing

trace-cmd -p function -g SyS_read

Ftrace no trace (set_ftrace_notrace)

trace -p nop -n schedule

Ftrace function stack trace ( options/func_stack_trace )

trace-cmd start -p function -l schedule --func-stack

Ftrace Events list

trace-cmd list -e

Set Event

trace-cmd start -p -e sched

Set Event pid

trace-cmd record -e syscalls -e exceptions -F echo  ftrace

Show output

trace-cmd show
trace-cmd report

source

https://www.kernel.org/doc/Documentation/trace/ftrace.txt	
https://www.kernel.org/doc/Documentation/trace/events.txt
https://www.kernel.org/doc/Documentation/trace/tracepoints.txt
https://www.kernel.org/doc/Documentation/trace/ftrace-design.txt
https://kernel-recipes.org/en/2017/talks/understanding-the-linux-kernel-via-ftrace/

Understanding Signals

2017-11-28T15:20:23+00:00

Signals are software interrupts to handle asynchronous events.

A signal may be sent from the kernel to a process, from a process to another process, or from a process to itself, or is sent either to a specific thread (Linux task) or to the process as a whole (Linux thread group).

In Linux, every signal has a name that begins with characters SIG.

When the signal is delivered, either it’s caught or ignored by a user handler or it has a default action.

Ignore - nothing will happen but SIGKILL and SIGSTOP cannot be ignored.
Catch and handle Signal Kernel will suspend execution of the process and run handle code or chose to terminate.
Default action, this could be the default action to terminate the process or ignore.

Signals Identifiers

Signals are defined in signal.h header file.

+--------------------+------------------+
|  POSIX signal      |  default action  |
+--------------------+------------------+
|  SIGHUP            |  terminate       |
|  SIGINT            |	terminate       |
|  SIGQUIT           |	coredump        |
|  SIGILL            |	coredump        |
|  SIGTRAP           |	coredump        |
|  SIGABRT/SIGIOT    |  coredump        |
|  SIGBUS            |	coredump        |
|  SIGFPE            |	coredump        |
|  SIGKILL           |	terminate(+)    |
|  SIGUSR1           |	terminate       |
|  SIGSEGV           |	coredump        |
|  SIGUSR2           |	terminate       |
|  SIGPIPE           |	terminate       |
|  SIGALRM           |	terminate       |
|  SIGTERM           |	terminate       |
|  SIGCHLD           |	ignore          |
|  SIGCONT           |	ignore(*)       |
|  SIGSTOP           |	stop(*)(+)      |
|  SIGTSTP           |	stop(*)         |
|  SIGTTIN           |	stop(*)         |
|  SIGTTOU           |	stop(*)         |   
|  SIGURG            |	ignore          |
|  SIGXCPU           |	coredump        |
|  SIGXFSZ           |	coredump        |
|  SIGVTALRM         |	terminate       |
|  SIGPROF           |	terminate       |
|  SIGPOLL/SIGIO     |	terminate       |
|  SIGSYS/SIGUNUSED  |	coredump        |
|  SIGSTKFLT         |	terminate       |
|  SIGWINCH          |	ignore          |
|  SIGPWR            |	terminate       |
|  SIGRTMIN-SIGRTMAX |	terminate       |
+--------------------+------------------+
|  non-POSIX signal  |  default action  |
+--------------------+------------------+
|  SIGEMT            |  coredump        |
+--------------------+------------------+

ignore - Nothing Happens
terminate - kill the process, i.e. all threads in the group, similar to exit_group.  The group leader (only) report WIFSIGNALED status to its parent.
coredump - write a core dump file describing all threads using the same Memory Management and then kill all those threads
stop - stop all the threads in the group, i.e. TASK_STOPPED state

Note that SIGKILL and SIGSTOP cannot be caught, blocked, or ignored.

Example how to catch a signal.

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <signal.h>

static void signal_handler (int singnu)
{
	printf("Caught SIGTERM\n");
	exit (EXIT_SUCCESS);
}

int main (int argc, char *argv[]) 
{
	if (signal (SIGTERM, signal_handler) == SIG_ERR) {
		printf("Cannot handle signal\n");
		exit(EXIT_FAILURE);
	}
	
	for (;;)
		pause();
	
	return 0;
}

sources

http://www.tldp.org/LDP/tlk/kernel/processes.html
http://elixir.free-electrons.com/linux/v4.15-rc2/source/include/linux/signal.h

Understanding Daemon Process

2017-11-17T15:20:23+00:00

A daemon process is a process which runs in background and has no controlling terminal. Normally daemon process are running on boot, running as root or other system user and handle system tasks.

The requirement to run a daemon it must run as a child of init process and not connected to a terminal.

How to create a daemon

Call fork() off the parent process.
Call exit() on parent process
Call setsid() to give new process group and session ID (SID) to become child process as a session leader.
Call signal() to ignore any signal sent from child to parent
Call again fork() to check that the daemon is not a session leader, which prevents it from acquiring a controlling terminal.
Call umask(0); Set new file permissions
Call chdir() to change the current working directory
Close file descriptors.
Reopen all file descriptor and redirect them to /dev/null, call open() /dev/null  for stdin to return a file descriptor and the dublicate with dup() for stdout and stderr.
Call daemon code.

Example code

#include <sys/types.h>
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <errno.h>
#include <unistd.h>
#include <signal.h>
#include <linux/fs.h>

int main(void) {

    pid_t pid;
    int i;

    pid = fork();

    if (pid == -1) 
    {
        return -1;
    }
    
    if (pid != 0) 
    {
        exit(EXIT_SUCCESS);
    }
    
    if (setsid() == -1)
    {
        return -1;
    }
    
    signal(SIGCHLD, SIG_IGN);

    pid = fork();

    if (pid < 0) 
    {
        exit(EXIT_FAILURE);
    }

    if (pid > 0) 
    {
        exit(EXIT_SUCCESS);
    }

    umask(0);

    if ((chdir("/")) == -1) 
    {
	    return -1;
    }

    for (i = sysconf(_SC_OPEN_MAX); i > 0; i--) 
    {
        close(i);
    }

    open("/dev/null", O_RDWR);
    dup(0);
    dup(0);

    while (1) 
    {
        /* run daemon code */
    }
    
    exit(EXIT_SUCCESS);
}

Other way on Linux systems you can call daemon() to create it.

       int daemon(int nochdir, int noclose);

https://stackoverflow.com/questions/17954432/creating-a-daemon-in-linux/17955149#17955149
https://linux.die.net/man/2/setsid
http://man7.org/linux/man-pages/man2/umask.2.html
http://man7.org/linux/man-pages/man3/sysconf.3.html
http://man7.org/linux/man-pages/man2/open.2.html
http://man7.org/linux/man-pages/man2/dup.2.html

Understanding Zombie Process

2017-11-17T15:15:51+00:00

Zombie Process is a state of process which has completed execution but still has an entry in the process table. These process have irresponsible parent process. The responsible Process table is a data structure in Linux kernel, that stores information about all current process.

The process table contain the PID, User, Scheduling value, Virtual memory, State, CPU, Memory etc.

Location of process is /proc and you can use ps, top, htop, pstree and many other tools to see the process or you can get the process table with a kernel module.

Example

#include <linux/module.h>
#include <linux/printk.h>
#include <linux/sched.h>

static int __init proc_list(void)
{
    struct task_struct *task;

    for_each_process(task)
        pr_info("%s [%d]\n", task->comm, task->pid);

    return 0;
}

static void __exit proc_exit(void)
{
	pr_info("Cleaning up");
}

module_init(proc_list);
module_exit(proc_exit);

An easy way to create a zombie process is to run fork and exit the child immediately and the parent will sleep and will not be informed about the exit. When a process exit(), all of the memory and resources associated with it are deallocated. Further note that if the parent process dies before the child, the kernel will reparent the child process to init.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main()
{
    pid_t child;

    if ((child = fork()) < 0)
        exit(1);

    if (child == 0)
        exit(0);

    sleep(120);

    return 0;
}

The parent process use wait() to get info about the child state.

if ( waitpid(child, &status, 0) == -1 ) {
    return EXIT_FAILURE;
    }

To get information if the process exit() correctly

if (WIFEXITED (status))
    return WEXITSTATUS(status);

To fix and remove zombie process entry from process table we need to inform parent process that child called exit.

We need to make the parent to wait() the status of child so it can report the exit of the child.

First we need to find the process parent id and zombie id, then we need to use gdb to attach on parent and call waitpid within the zombie id.

Example

ps axo stat,ppid,pid,comm | grep -w defunct
gdb  
attach <process id of parent process>
call waitpid (<zombie id>,0,0)
detach

sources

https://linux.die.net/man/2/waitpid:
https://en.wikipedia.org/wiki/Zombie_process

Limiting syscalls with seccomp

2017-09-20T14:15:51+00:00

Seccomp and Seccomp-BPF are used to limit the system calls available to a Linux process. System call filtering isn’t a sandbox. It provides a clearly defined mechanism for minimizing the exposed kernel surface.

The initial implementation, also known as “mode 1 seccomp” only allowed ‘read’, ‘write’, ‘exit’ and ‘sigreturn’ syscalls to be only possible to read/write to already opened files and exit.

Example how we call seccomp:

#include <stdio.h>
#include <stdlib.h>
#include <sys/prctl.h>
#include <linux/seccomp.h>
#include <unistd.h>
int main(int argc, char* argv[])
{
    prctl(PR_SET_SECCOMP, SECCOMP_MODE_STRICT);
    
    /* Redirect stderr to stdout */
    dup2(1, 2);
    printf("Nothing.\n");
  
  return 0; 
}

Seccomp is good for absolute restrictions, a more fine grained approach is required when attempting to lock down more complex applications. In order to solve this problem Seccomp - Berkley Packet Filter (Seccomp-BPF) was introduced.

Seccomp-BPF program receives the following struct as an input argument. We can filter based on the system call number and on the arguments.

struct seccomp_data {
    int nr;                     /* System call number */
    __u32 arch;                 /* AUDIT_ARCH_* value  (see <linux/audit.h>)*/
    __u64 instruction_pointer;  /* CPU instruction pointer */
    __u64 args[6];              /* Up to 6 system call arguments */
    };

Seccomp using Linux Socket Filtering aka Berkeley Packet Filter (BPF) which contains the following structures

struct sock_filter {/* Filter block */
    __u16code;   /* Actual filter code */
    __u8jt;      /* Jump true */
    __u8jf;      /* Jump false */
    __u32k;      /* Generic multiuse field */
    };

struct sock_fprog { /* Required for SO_ATTACH_FILTER. */
    unsigned short len;     /* Number of filter blocks */
    struct sock_filter __user *filter;
    };

So basically what BPFs do in seccomp is to operate on this data, and return a value that tells the kernel what to do next: allow the process to perform the call (SECCOMP_RET_ALLOW), kill it (SECCOMP_RET_KILL), or other options.

As per the documentation of seccomp the best practice is to check the arch system call numbers with seccomp_data->arch to avoid any issues like if the syscall numbers in the different calling conventions overlap, then checks in the filters may be abused.

BPF_STMT(BPF_LD | BPF_W | BPF_ABS, (offsetof(struct seccomp_data, arch))),
BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, AUDIT_ARCH_X86_64, 1, 0),
BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_KILL),

Example filter to allow open syscall

struct sock_filter filter [] = {
    BPF_STMT(BPF_LD+BPF_W+BPF_ABS, offsetof(struct seccomp_data, nr)),
    BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_open, 0, 1),
    BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
    BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_KILL),
    };

PR_SET_NO_NEW_PRIVS, which impedes child processes to have more privileges than those of the parent. This is needed to make the following call to prctl, which sets the seccomp filter using the PR_SET_SECCOMP option, succeed even when not being root.

if (prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) == -1) {
    perror("setting new privileges");
    return 1;
    }

Set seccomp

if (prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &prog) == -1) {
    perror("seccomp filter error");
    return 1;
    }

Simple example

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <stddef.h>
#include <sys/prctl.h>
#include <linux/seccomp.h>
#include <linux/filter.h>
#include <linux/unistd.h>
#include <linux/audit.h>

int main(int argc, char* argv[])
{
    int fd;
    
    #define syscall_nr (offsetof(struct seccomp_data, nr))
    #define arch_nr (offsetof(struct seccomp_data, arch))
    
    /* architecture x86_64 */
    #define REG_SYSCALL REG_RAX
    #define ARCH_NR AUDIT_ARCH_X86_64

    struct sock_filter filter[] = {
        /* Validate architecture. */
        BPF_STMT(BPF_LD+BPF_W+BPF_ABS, arch_nr),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, ARCH_NR, 1, 0),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_KILL),
        /* Get system call number. */
        BPF_STMT(BPF_LD+BPF_W+BPF_ABS, syscall_nr),
        /* List allowed syscalls. */
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_rt_sigreturn, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_exit_group, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_exit, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_read, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_write, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_open, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_dup, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_fcntl, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_brk, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_fstat, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_close, 0, 1),
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
        /* Kill Process */
        BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_KILL),
        };
    
    struct sock_fprog prog = {
	.len = (unsigned short)(sizeof(filter)/sizeof(filter[0])),
	.filter = filter,
        };

    if ( prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) == -1 ) {
        perror("prctl(PR_SET_NO_NEW_PRIVS)\n");
        return -1;
        }
    
    if ( prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &prog) == -1 ) {
        perror("Seccomp filter error\n");
        return -1;
        }
    
    if ( fd = open("/etc/passwd", O_RDONLY) == -1 ){
        perror("Error");
        return -1;
        }
    
    close(fd);

    return 0;
 
}

sources

https://outflux.net/teach-seccomp/
http://www.alfonsobeato.net/c/filter-and-modify-system-calls-with-seccomp-and-ptrace/
https://www.illogicalexpressions.com/linux/2016/08/31/seccomp-and-seccomp-bpf.html
https://www.kernel.org/doc/Documentation/networking/filter.txt
https://www.kernel.org/doc/Documentation/prctl/seccomp_filter.txt
http://elixir.free-electrons.com/linux/latest/source/include/uapi/linux/seccomp.h#L47