Post: Inter Process Communication (IPC)

In this post I will cover the various inter-process communication (IPC) methods and discuss their trade-offs.

Signal

Overview: Asynchronous (sender don’t need to coordinate with the receiver) notification system between process.

Functionality:

• Notification:
  1. Signal can be sent to a process or thread to notify an event
  2. When a signal is sent, the OS will interrupt the normal flow of execution.
  3. If the signal has previously registered signal handler, the signal handler will be executed otherwise the default signal handler will be executed.
• Block/Unblocking signals
  • Use sigprocmask to block signals. Blocked signals will not be delivered to the process until they are unblocked. SIGKILL and SIGSTOP cannot be unblocked.

Pros:

1. Lightweight IPC with little overhead

Cons:

1. Vulnerable to race conditions. A signal could be delivered to a process when it is already executing a signal handler.
2. Could interrupt system calls
3. Signal handlers cannot have side effects or non-reentrant functions
   • non-reentrant functions are functions that will result in race condition if a function is interrupted mid-way through execution and called again even though the first function has not completed.
4. Mitigation:
   • Signals are pushed into a queue in the handler and immediately returned. Main handling of signals will occur outside of the signal handler.

Sockets / Unix Domain Sockets

• Process can communicate via the network stack through IP/TCP
  • Each socket (unique REMOTE-IP, REMOTE-PORT, CLIENT-IP, CLIENT-PORT) is represented by a file descriptor
• On UNIX, provides UNIX domain socket that allows for TCP-like communication through the kernel instead of the network stack.

Message Queue

Overview: UNIX OS provides a message queue (pub-sub pattern) API for IPC.

Notes:

• Uses direct/in-direct sys calls

Steps to use message queue:

• mq_open
  • Can be set to get or create the message queue
  • Returns the file descriptor to the message queue
  • Settings available:
    1. Maximum number of messages
    2. Maximum size of each message
    3. Set if the queue is blocking or not
• mq_send
  • With the file descriptor of the message queue, sends the message to it
  • If the queue is full
    • Blocking set: blocks until the queue has available space
    • Non-blocking set: returns an error
  • Able to set the priority of the message
• mq_receive
  • Receives the message from the message queue
  • Can be blocking when the queue is empty if the queue is created with blocking

Pros:

• Built in thread-safe message queue system
  • Setting MT-thread safe
• Allows for blocking or non-blocking communication
• By default receiving message will block

Cons:

• Limited to only pub-sub pattern
• Not lightweight: provides priority and message types
• Does not work if you want to IPC through distributed systems

Pipes vs Message Queue:

• Pipes are unbounded (unlimited stream of bytes) but message queue are bounded (messages have maximum size)
• Both pipes and allows for multiple pub and sub

Pipes

Overview: one way FIFO communication channel (producer and consumer driven)

Steps to use pipes:

• Anonymous pipes in C
  • Call pipe with an int array of size 2
    • The read fd will be in the first element and write fd will be in the second element
• Named pipes
  • Call mknod to create pipe
    • Takes a pipe id /...
  • The program treats the pipes as a normal files. Use write to write to the pipe and read to read from the pipe

Anonymous Pipes:

• Used by executing process with |
• Pipes the stdout file descriptor of the LHS process to the stdin file RHS process
• Only last during the duration of the process

Name pipes:

• Instead of having | to specify the pipe, create a pipe on the file system to allow process to explicitly refer to it.

Pros:

• If the consumer (RHS) close the pipes, the LHS will receive SIGPIPE signal when trying to write to the pipe and will be killed by the signal handler.

Shared Memory:

Overview: multiple process can share the same memory address in physical memory

Steps to create shared memory

• shm_open:
  • With an identifier, create a new or open an existing shared memory object
  • Returns a file descriptor to the shared memory object
• ftruncate:
  • Set the size of shared memory object
  • Takes in the file descriptor that is returned from shm_open
• mmap:
  • Maps the virtual address space of the process to the memory address to the shared memory
  • Mainly takes in the file descriptor of the shared memory object
  • This will allow the process to access the data at the shared memory

Parent and Child:

• If the parent process set ups the shared memory before forking, the child process will inherrit all mmap regions.

  Child processes inherit the address space and all mapped regions of the parent process. Once the object is opened, the child process can map it with the mmap function to establish a map reference. If the object is already mapped, the child process also inherits the mapped region.

• The shared memory in that the child copies is the same as the shared memory in the parent process

Pros:

• Allow for synchronization primitives (shared mutex)

Cons:

• Memory access by the different process can vary in non-uniform memory access (NUMA)
• Could result in false sharing: additional unnecessary cache coherence protocol overhead.

Memory-mapped file:

Overview: A segment of virtual memory that has been assigned byte-for-byte with some file or file-like (represented using file descriptor). Instead of each representing a frame in physical memory it will represent the file on disk.

IPC with mmap:

• Process can communicate through writing.

Cons:

• Could result in bugs with compiler optimisation (need to use volatile qualifier in c++) removing redundant read-write to a memory