Tag Archives: linux

C++ || Snippet – How To Use Message Queues For Interprocess Communication

The following is sample code which demonstrates the use of the msgsnd, msgrcv, and msgget function calls for use with message queues on Unix based systems.

Message queues are used for interprocess communication. They provide two (or more) separate programs (i.e: sender and receiver) with the ability to interact with eachother. This is ideal because the sender and receiver do not need to interact with the message queue at the same time. Messages can be sent at one point in time, be placed on the queue until the receiver is ready for it, and then be removed at another point in time when the program(s) that service the queue are finally ready to receive a message.

This concept is different from threaded and forked process IPC procedures, which often times process data in a more streamlined manner. Rather than following a strict pattern as to when data is to be sent and received, queuing is a mechanism in which messages are held until the receiving application is ready to process them.

The example on this page demonstrates the above use of a message queue to transfer data between two separate programs. The sending program (client.cpp) sends an integer and string variable to the receiving program (server.cpp), which then displays that received data to the screen.

NOTE: The server program must be ran before the client program!

=== 1. Server.cpp (Receiver) ===

=== 2. Client.cpp (Sender) ===

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

You can view all allocated message queues using the ipcs command. You can delete a message queue from command line using ipcrm -q [KEY SHOWN BY IPCS]. Message queues are a finite resource. If something goes wrong during the execution of your program, you must manually delete all your queues.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output.

SERVER OUTPUT:

The server has started!

Waiting for someone to connect to server id #753664 with the key 1258295474

someNumber = 0 buff = Message queues are awesome!
someNumber = 1 buff = Message queues are awesome!
someNumber = 2 buff = Message queues are awesome!
someNumber = 3 buff = Message queues are awesome!
someNumber = 4 buff = Message queues are awesome!
someNumber = 5 buff = Message queues are awesome!
someNumber = 6 buff = Message queues are awesome!
someNumber = 7 buff = Message queues are awesome!
someNumber = 8 buff = Message queues are awesome!
someNumber = 9 buff = Message queues are awesome!

Server is now shutting down!

CLIENT OUTPUT:

Successfully connected to server id #753664 with the key 1258295474

Now sending messages....Sending complete!

C++ || Multi-Hash Interprocess Communication Using Fork, Popen, & Pipes

The following is another homework assignment which was presented in an Operating Systems Concepts class. Using two pipes, the following is a program which implements the computing of hash values on a file using the MD5, SHA1, SHA224, SHA256, SHA384, and SHA512 hashing algorithms provided on Unix based systems.

REQUIRED KNOWLEDGE FOR THIS PROGRAM

How To Use Fork
How To Use Pipe
How To Use Popen

==== 1. OVERVIEW ====

Hash algorithms map large data sets of variable length (e.g. files), to data sets of a fixed length. For example, the contents of a 1GB file may be hashed into a single 128-bit integer. Many hash algorithms exhibit an important property called an avalanche effect – slight changes in the input data trigger significant changes in the hash value.

Hash algorithms are often used for verifying the integrity of files downloaded from the WEB. For example, websites hosting a file usually post the hash value of the file using the MD5 hash algorithm. By doing this, the user can then verify the integrity of the downloaded file by computing the MD5 algorithm on their own, and compare their hash value against the hash value posted on the website. The user will know if the download was valid only if the two hash values match.

==== 2. TECHNICAL DETAILS ====

The following implements a program for computing the hash value of a file using the MD5, SHA1, SHA224, SHA256, SHA384, and SHA512 hashing algorithms provided on Unix based systems.

This program takes the name of the target file being analyzed as a command line argument, and does the following:

1. Check to make sure the file exists.
2. Create two pipes.
3. Create a child process.
4. The parent transmits the name of the file to the child (over the first pipe).
5. The child receives the name of the file and computes the hash of the file using the MD5 algorithm (using Linux program md5sum).
6. The child transmits the computed hash to the parent (over the second pipe) and terminates.
7. The parent receives the hash, prints it, and calls wait().
8. Repeat the same process starting with step 3, but using algorithms SHA1...SHA512.
9. The parent terminates after all hashes have been computed.

The use of the popen function is used in order to launch the above programs and capture their output into a character array buffer.

This program also uses two pipes. The two pipes created are the following:

(1) Parent to child pipe: Used by the parent to transfer the name of the file to the child. The parent writes to this pipe and the child reads it.

(2) Child to parent pipe: Used by the child to transfer the computed hashes to the parent. The child writes to this pipe and the parent reads it.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

Using the following example input file located here, the following is sample output:

C++ || Snippet – How To Use Popen & Save Results Into A Character Array

The following is sample code which demonstrates the use of the “popen” function call on Unix based systems.

The “popen” function call opens a process by creating a pipe, forking, and invoking the shell. It does this by executing the command specified by the incoming string function parameter. It creates a pipe between the calling program and the executed command, and returns a pointer to a stream that can be used to either read from or write to the pipe.

The following example demonstrates how to save the results of the popen command into a char array.

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:


Program output: fa97c59cc414e42d4e0e853ddf5b4745 /bin/ls

C++ || Serial & Parallel Multi Process File Downloader Using Fork & Execlp

The following is another homework assignment which was presented in an Operating Systems Concepts class. The following are two multi-process programs using commandline arguments, which demonstrates more practice using the fork() and execlp() system calls on Unix based systems.

==== 1. OVERVIEW ====

File downloaders are programs used for downloading files from the Internet. The following programs listed on this page implement two distinct type of multi-process downloaders:

1. a serial file downloader which downloads files one by one.
2. a parallel file downloader which dowloads multiple files in parallel.

In both programs, the parent process first reads a file via the commandline. This file which is read is the file that contains the list of URLs of the files to be downloaded. The incoming url file that is read has the following format:

[URL1]
[URL2]
.
.
.
[URLN]

Where [URL] is an http internet link with a valid absolute file path extension.
(i.e: http://newsimg.ngfiles.com/270000/270173_0204618900-cc-asmbash.jpg)

After the url file is parsed, next the parent process forks a child process. Each created child process uses the execlp() system call to replace its executable image with that of the “wget” program. The use of the wget program performs the actual file downloading.

==== 2. SERIAL DOWNLOADER ====

The serial downloader downloads files one at a time. After the parent process has read and parsed the incoming url file from the commandline, the serial downloader proceeds as follows:

1. The parent forks off a child process.
2. The child uses execlp("/usr/bin/wget", "wget", [URL STRING1], NULL) system call in order to replace its program with wget program that will download the first file in urls.txt (i.e. the file at URL ).
3. The parent executes a wait() system call until the child exits.
4. The parent forks off another child which downloads the next file specified in url.txt.
5. Repeat the same process until all files are downloaded.

The following is implemented below:

Since the serial downloader downloads files one at a time, that can become very slow. That is where the parallel downloader comes in handy!

==== 3. PARALLEL DOWNLOADER ====

The parallel downloader downloads files all at once and is implemented much like the serial downloader. The parallel downloader proceeds as follows:

1. The parent forks off n children, where n is the number of URLs in url.txt.
2. Each child executes execlp("/usr/bin/wget", "wget", [URL STRING], NULL) system call where each is a distinct URL in url.txt.
3. The parent calls wait() (n times in a row) and waits for all children to terminate.
4. The parent exits.

The following is implemented below:


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

Also note, while the parallel downloader executes, the outputs from different children may intermingle.

C++ || Snippet – How To Use Fork & Pipe For Interprocess Communication

The following is sample code which demonstrates the use of the fork, read, and write function calls for use with pipes on Unix based systems.

A pipe is a mechanism for interprocess communication. Data written to a pipe by one process can be read by another process. Creating a pipe is achieved by using the pipe function, which creates both the reading and writing ends of the pipe file descriptor.

In typical use, a parent process creates a pipe just before it forks one or more child processes. The pipe is then used for communication between either the parent or child processes, or between two sibling processes.

A real world example of this kind of communication can be seen in all operating system terminal shells. When you type a command in a shell, it will spawn the executable represented by that command with a call to fork. A pipe is opened to the new child process, and its output is read and printed by the terminal.


QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

Parent is forking a child.
Parent is now waiting for child id #12776 to complete..

Starting the child process..

Message from the parent via the pipe: Greetings From Your Parent!

Program is now exiting...

The child process is complete and has terminated!

Program is now exiting...

C++ || Snippet – How To Use Fork & Execlp For Interprocess Communication

The following is sample code which demonstrates the use of the “fork” and “execlp” function calls on Unix based systems.

The “fork” function call creates a new process by duplicating the calling process; or in more simpler terms, it creates a duplicate process (a child) of the calling (parent) process.

This new process, referred to as the child, is an exact duplicate of the calling process, referred to as the parent.

The “execlp” function call is part of a family of functions which replaces a current running process image with a new process image. That means by using the “execlp” function call, you are basically replacing the entire current running process with a new user defined program.

Though fork and execlp are not required to be used together, they are often used in conjunction with one another as a way of creating a new program running as a child of another process.

QUICK NOTES:
The highlighted lines are sections of interest to look out for.

The code is heavily commented, so no further insight is necessary. If you have any questions, feel free to leave a comment below.

The following is sample output:

Parent is forking a child.
Parent is now waiting for child id #10872 to complete..

Starting the child process..

total 1466
-rw-r--r-- 1 admin admin 468 Apr 27 2012 nautilus-computer.desktop
-rwxrwxr-x 1 admin admin 9190 Aug 19 15:17 ForkExample
-rw-rw-r-- 1 admin admin 1640 Aug 19 15:17 ForkExample.cpp

The child process is complete and has terminated!

Parent is now exiting...

Python || How To Install PyPdf Using Python 3

pyPdf” is a pure Python library built as a PDF toolkit. It is capable of:

• Extracting document information (title, author, ...),
• Splitting documents page by page,
• Merging documents page by page,
• Cropping pages,
• Merging multiple pages into a single page,
• Encrypting and decrypting PDF files.

By being pure Python, it should run on any Python platform without any dependencies on external libraries. It can also work entirely on StringIO objects rather than file streams, allowing for PDF manipulation in memory. It is therefore a useful tool for websites that manage or manipulate PDFs.

=== 1. INSTALLING PYPDF ===

A Windows installer for pyPdf is available here. At the time of this writing, the installer that was listed on the download page was titled “pyPdf-1.13.win32.exe.”

Note: If you are using Python 2 and want to install pyPdf, the Windows installer available on the download page should be all you need. No further installation instructions are necessary.

Next, follow the command prompts from the installer and wait!

When the pyPdf installer is completed, you should see the newly installed files which correspond to this module located in the following directory:


C:\Python32\Lib\site-packages\pyPdf

Note: Python 3.2 is installed on my system, and the above directory reflects that. The Python directory may be named something different on your individual system depending on the Python version you are using.

If everything installed correctly, proceed to the next step.

=== 2. UPDATE EXISTING PYPDF FILES ===

pyPdf was originally written for Python 2, but a Python 3 compatible branch has since been made available. The updated files can be found here, and enable pyPdf to be integrated with Python 3.

To update these new Python 3 files with the old Python 2 files, locate the following directory on your system:


C:\Python32\Lib\site-packages\pyPdf

Here is a sample screenshot demonstrating the files which resides in the above directory: (click to enlarge)

Next, update all of the old Python 2 files that’s listed in the above directory with the new Python 3 files that’s listed on this page.

One way to update the files that’s currently stored on your computer is to locate the file on your system with the exact same file name as listed here, and copy/paste the contents of the new file into the contents of the old file that’s currently stored on your computer.

Or you can download all of the files at once here, and move/replace these new files with the existing files that’s currently located on your system.

=== 3. UBUNTU USERS ===

If you are an Ubuntu Linux user and want to install pypdf, open the terminal and run the following command:

NOTE: Replace “python3.2” with whatever version of python that’s installed on your system.


sudo mkdir /usr/local/lib/python3.2/dist-packages/pyPdf


Next, copy and paste the files located in the above download link into the following directory:


/usr/local/lib/python3.2/dist-packages/pyPdf


Once the following steps are completed, you should now be ready to use pyPdf with Python 3 programs!