Thursday, November 12, 2009

Chapter 21. Project: Internet Radio



[ Team LiB ]






Chapter 21. Project: Internet Radio


Broadcast, telephone and network technologies are converging rapidly, blurring the distinction between telephone and television. Software for video and telephone conferencing on the Internet is widely available, and most cable companies now offer high-speed Internet connections. Telephone companies have entered the entertainment business with video-on-demand and content services. The final resolution of these competing forces will probably be determined by politics and regulatory decisions as well as by technical merit. Whatever the outcome, more computers will handle voice, audio and video streams in addition to data. This chapter explores how well network communication protocols such as UDP, multicast and TCP support streaming media applications. The chapter outlines a project to send audio streams over a network under various conditions. The project explores timing, buffering and packet loss, as well as synchronization and a dynamic number of receivers.



Objectives


  • Learn about streaming media

  • Experiment with UDP, multicast and TCP

  • Explore timing strategies for multimedia

  • Use audio in a real application

  • Understand synchronization with multiple receivers







    [ Team LiB ]



    22.2 Server Architectures



    [ Team LiB ]






    22.2 Server Architectures


    Chapter 18 introduced three models of client-server communication: the serial-server (Example 18.2), the parent-server (Example 18.3), and the threaded-server (Example 18.6), respectively. Because the parent-server strategy creates a new child process to handle each client request, it is sometimes called process-per-request. Similarly, the threaded-server strategy creates a separate thread to handle each incoming request, so it is often called the thread-per-request strategy.


    An alternative strategy is to create processes or threads to form a worker pool before accepting requests. The workers block at a synchronization point, waiting for requests to arrive. An arriving request activates one thread or process while the rest remain blocked. Worker pools eliminate creation overhead, but may incur extra synchronization costs. Also, performance is critically tied to the size of the pool. Flexible implementations may dynamically adjust the number of threads or processes in the pool to maintain system balance.



    Example 22.1

    In the simplest worker-pool implementation, each worker thread or process blocks on the accept function, similar to a simple serial server.



    for ( ; ; ) {
    accept request
    process request
    }


    Although POSIX specifies that accept be thread-safe, not all operating systems currently support thread safety. Alternatively, workers can block on a lock that provides exclusive access to accept, as the next example shows.



    Example 22.2

    The following worker-pool implementation places the accept function in a protected critical section so that only one worker thread or process blocks on accept at a time. The remaining workers block at the lock or are processing a request.



    for ( ; ; ) {
    obtain lock (semaphore or mutex)
    accept request
    release lock
    process request
    }


    POSIX provides semaphores for interprocess synchronization and mutex locks for synchronization within a process.



    Exercise 22.3

    If a server uses N workers, how many simultaneous requests can it process? What is the maximum number of simultaneous client connections?


    Answer:


    The server can process N requests simultaneously. However, additional client connections can be queued by the network subsystem. The backlog parameter of the listen function provides a hint to the network subsystem on the maximum number of client requests to queue. Some systems multiply this hint by a fudge factor. If the network subsystem sets its maximum backlog value to B, a maximum of N + B clients can be connected to the server at any one time, although only N clients may be processed at any one time.



    Another worker-pool approach for threaded servers uses a standard producer-consumer configuration in which the workers block on a bounded buffer. A master thread blocks on accept while waiting for a connection. The accept function returns a communication file descriptor. Acting as the producer, the master thread places the communication file descriptor for the client connection in the bounded buffer. The worker threads are consumers that remove file descriptors and complete the client communication.


    The buffer implementation of the worker pool introduces some interesting measurement issues and additional parameters. If connection requests come in bursts and service time is short, buffering can smooth out responses by accepting more connections ahead than would be provided by the underlying network subsystem. On the other hand, if service time is long, accepted connections languish in the buffer, possibly triggering timeouts at the clients. The number of additional connections that can be accepted ahead depends on the buffer size and the order of the statements synchronizing communication between the master producer and the worker consumers.



    Exercise 22.4

    How many connections ahead can be accepted for a buffer of size M with a master and N workers organized as follows?



    Master:
    for ( ; ; ) {
    obtain a slot
    accept connection
    copy the file descriptor to slot
    signal item
    }

    Worker:
    for ( ; : ) {
    obtain an item (the file descriptor)
    process the communication
    signal slot
    }

    Answer:


    If N M, then each worker holds a slot while processing the request, and the master cannot accept any connections ahead. For N < M the master can process M � N connections ahead.




    Exercise 22.5

    How does the following strategy differ from that of Exercise 22.4? How many connections ahead can be accepted for a buffer of size M with a master and N workers organized as follows?



    Master:
    for ( ; ; ) {
    accept connection
    obtain a slot
    copy the file descriptor to slot
    signal item
    }

    Worker:
    for ( ; ; ) {
    obtain an item (a file descriptor)
    signal slot
    process the communication
    }

    Answer:


    The strategy here differs from that of Exercise 22.4 in two respects. First, the master accepts a connection before getting a slot. Second, each worker thread immediately releases the slot (signal slot) after copying the communication file descriptor. In this case, the master can accept up to M+1 connections ahead.




    Exercise 22.6

    In what way do system parameters affect the number of connections that are made before the server accepts them?


    Answer:


    The backlog parameter set by listen determines how many connections the network subsystem queues. The TCP flow control mechanisms limit the amount that the client can send before the server calls accept for that connection. The backlog parameter is typically set to 100 or more for a busy server, in contrast to the old default value of 5 [115].




    Exercise 22.7

    What a priori advantages and disadvantages do worker-pool implementations have over thread-per-request implementations?


    Answer:


    For short requests, the overhead of thread creation and buffer allocation can be significant in thread-per-request implementations. Also, these implementations do not degrade gracefully when the number of simultaneous connections exceeds system capacity�these implementations usually just keep accepting additional connections, which can result in system failure or thrashing. Worker-pool implementations save the overhead of thread creation. By setting the worker-pool size appropriately, a system administrator can prevent thrashing and crashing that might occur during busy times or during a denial-of-service attack. Unfortunately, if the worker-pool size is too low, the server will not run to full capacity. Hence, good worker-pool deployments need the support of performance measurements.




    Exercise 22.8

    Can the buffer-pool approach be implemented with a pool of child processes?


    Answer:


    The communication file descriptors are small integer values that specify position in the file descriptor table. These integers only have meaning in the context of the same process, so a buffer-pool implementation with child processes would not be possible.



    In thread-per-request architectures, the master thread blocks on accept and creates a thread to handle each request. While the size of the pool limits the number of concurrent threads competing for resources in worker pool approaches, thread-per-request designs are prone to overallocation if not carefully monitored.



    Exercise 22.9

    What is a process-per-request strategy and how might it be implemented?


    Answer:


    A process-per-request strategy is analogous to a thread-per-request strategy. The server accepts a request and forks a child (rather than creating a thread) to handle it. Since the main thread does not fork a child to handle the communication until the communication file descriptor is available, the child inherits a copy of the file descriptor table in which the communication file descriptor is valid.


    The designs thus far have focused on the communication file descriptor as the principal resource. However, heavily used web servers are often limited by their disks, I/O subsystems and memory caches. Once a thread receives a communication file descriptor and is charged with handling the request, it must locate the resource on disk. This process may require a chain of disk accesses.




    Example 22.10

    The client request to retrieve /usp/exercises/home.html may require several disk accesses by the OS file subsystem. First, the file subsystem locates the inode corresponding to usp by reading the contents of the web server's root directory and parsing the information to find usp. Once the file subsystem has retrieved the inode for usp, it reads and parses data blocks from usp to locate exercises. The process continues until the file subsystem has retrieved the actual data for home.html. To eliminate some of these disk accesses, the operating system may cache inodes indexed by pathname.



    To avoid extensive disk accesses to locate a resource, servers often cache the inode numbers of the most popular resources. Such a cache might be effectively managed by a single thread or be controlled by a monitor.


    Disk accesses are usually performed through the I/O subsystem of the operating system. The operating system provides caching and prefetching of blocks. To eliminate the inefficiency of extra copying and blocking through the I/O subsystem, web servers sometimes cache their most popular pages in memory or in a disk area that bypasses the operating system file subsystem.






      [ Team LiB ]



      Querying Filesystem Usage










      Perl for System AdministrationSearch this book









      2.6. Querying Filesystem Usage








      Given the methods of controlling
      filesystem usage we've just explored, it is only natural to
      want to keep track of how well they work. To end this chapter,
      let's look at a method for querying the filesystem usage on
      each of the operating systems found in this book.








      MacOS is the operating system for which this
      task is hardest. MacOS does have a Macintosh Toolbox routine
      (PBHGetVInfo) to retrieve volume information,
      but at the current time there is no MacPerl module available to make
      calling this function easy. Instead, we have to take a roundabout
      approach and ask the Finder to query this
      information for us. This is easy in practice thanks to a glue module,
      but the setup needed for this method makes MacOS the more difficult
      operating system to deal with.







      All the materials for the following involve work by Chris Nandor and
      can be found at http://pudge.net
      or on CPAN. Bear with me as we go over this setup step by step:










      1. Install the
        cpan-mac bundle. cpan-mac includes the
        CPAN.pm module by Andreas J. K&#246;nig and other
        handy modules we mentioned in Chapter 1, "Introduction". Even if
        you don't want to query filesystem usage from MacOS,
        you'll still be well served by installing this bundle. When you
        install this bundle, be sure to follow all of the directions found in
        the README file.




      2. Install the latest Mac::AppleEvents::Simple module by dropping the distribution file on
        top of the installme droplet.




      3. Install the Mac::Glue
        module. The
        installme droplet decompresses and unpacks the
        contents of the Mac::Glue distribution file into a
        new folder as part of the installation process. Be sure to run the
        gluedialect and gluescriptadds
        setup scripts from the scripts subfolder of the
        unpacked distribution.




      4. Create the finder glue file. Open your System
        Folder
        and drag the Finder file on
        top of the gluemac droplet to create the necessary
        glue file (and, in a particularly nice touch by Nandor, to create pod
        documentation for the glue).




      This complex setup process allows us to write the following
      simple-looking code:








      use Mac::Glue qw(:all);

      $fobj = new Mac::Glue 'Finder';

      $volumename = "Macintosh HD"; # the name of one of our mounted disks
      $total = $fobj->get($fobj->prop('capacity',
      disk => $volumename),
      as => 'doub');
      $free = $fobj->get($fobj->prop('free_space',
      disk => $volumename),
      as => 'doub');

      print "$free bytes of $total bytes free\n";








      Let's move to easier territory.
      If we wanted to query the same information from a Win32 machine, we
      could use Dave Roth's Win32::AdminMisc
      module:








      use Win32::AdminMisc;

      ($total,$free) = Win32::AdminMisc::GetDriveSpace("c:\\");

      print "$free bytes of $total bytes free\n";








      Finally, let's end this chapter by
      looking at the Unix equivalent. There are several Unix modules
      available, including Filesys::DiskSpace by Fabien
      Tassin, Filesys::Df by Ian Guthrie, and
      Filesys::DiskFree by Alan R. Barclay. The first
      two of these make use of the Unix system call statvfs(
      )
      while the last one actually parses the output of the
      Unix command df on all of the systems it supports.
      Choosing between these modules is mostly a matter of personal
      preference and operating system support. I prefer
      Filesys::Df because it offers a rich feature set
      and does not spawn another process (a potential security risk, as
      discussed in Chapter 1, "Introduction") as part of a query.
      Here's one way to write code equivalent to the previous two
      examples:








      use Filesys::Df;

      $fobj = df("/");

      print $fobj->{su_bavail}*1024." bytes of ".
      $fobj->{su_blocks}*1024." bytes free\n";







      We have to do a little bit of arithmetic (i.e.,
      *1024) because Filesys::Df
      returns values in terms of blocks, and each block is 1024 bytes on
      our system. The df( ) function for this module
      can be passed a second optional argument for block size if necessary.
      Also worth noting about this code are the two hash values we've
      requested. su_bavail and
      su_blocks are the values returned by this module
      for the "real" size and disk usage information. On most
      Unix filesystems, the df command would show a
      value that hides the standard 10% of a disk set aside for superuser
      overflow. If we wanted to see the total amount of space available and
      the current amount free from a normal user's perspective, we
      would have used user_blocks and
      user_bavail instead.







      With the key pieces of Perl code we've just seen, it is
      possible to build more sophisticated disk monitoring and management
      systems. These filesystem watchdogs will help you deal with space
      problems before they occur.















      Copyright © 2001 O'Reilly & Associates. All rights reserved.







      21.6 Multicast Implementation of Radio Broadcasts



      [ Team LiB ]






      21.6 Multicast Implementation of Radio Broadcasts


      Copy UDPSendBcast.c into UDPSendMcast.c and compile it as UDPSendMcast. Modify UDPSendMcast to take a multicast address as an additional command-line argument. The port argument is now the multicast port for sending. The sender does not need to know anything about the receivers and does not have any direct contact with them. The sender's only responsibility is to send.


      Copy UDPRecvBcast.c into UDPRecvMcast.c and compile it as UDPRecvMcast. Modify UDPRecvMcast to receive audio from UDPSendMcast. The first command-line argument of UDPRecvMcast is a multicast address, and the second command-line argument is a multicast port. The UDPRecvMcast program now only receives messages and does not send anything over the network.



      Exercise 21.36

      How would you incorporate into the receiver the ability to display a message indicating how far along the audio transmission is when it joins?


      Answer:


      The receiver can estimate the time from first sequence number of the first audio packet that it receives, given that eight sequence numbers corresponds to one second of audio.




      Exercise 21.37

      How does UDPRecvMcast behave under the four basic test cases?


      Answer:


      UDPRecvMcast behaves as the other UDP implementations did. The receiver loses data if it is suspended long enough, and the receivers are independent.







        [ Team LiB ]



        10.2 Simple Timers



        [ Team LiB ]






        10.2 Simple Timers


        Operating systems often implement multiple software timers that are based on a single hardware timer. A software timer can be represented by a timer number and an indication of when the timer expires. The implementation depends on the type of hardware timer available.


        Suppose the hardware timer generates interrupts at regular short intervals called the clock tick time. The timer interrupt service routine monitors the time remaining on each timer (in terms of clock ticks) and decrements this time for each tick of the clock. When a timer decrements to 0, the program takes the appropriate action. This approach is inefficient if the number of timers is large or if the clock tick time is short.


        Alternatively, a program can keep the timer information in a list sorted by expiration time. Each entry contains a timer number and an expiration time. The first entry in the list contains the first timer to expire and the time until expiration (in clock ticks). The second entry contains the next timer to expire and the expiration time relative to the time the first timer expires, and so on. With this representation, the interrupt service routine decrements only one counter on each clock tick, but the program incurs additional overhead when starting a timer. The program must insert the new timer in a sorted list and update the time of the timer that expires immediately after the new one.



        Exercise 10.1

        For each of the two implementation approaches described above, what is the time complexity of the interrupt handler and the start timer function in terms of the number of timers?


        Answer:


        Suppose there are n timers. For the first method, the interrupt handler is O(n) since all timer values must be decremented. The start timer function is O(1) since a timer can be started independently of the other timers. For the second method, the interrupt handler is usually O(1) since only the first timer value must be decremented. However, when the decrement causes the first timer to expire, the next entry has to be examined to make sure it did not expire at the same time. This algorithm can degenerate to O(n) in the worst case, but in practice the worst case is unlikely. The start timer function is O(n) to insert the timer in a sorted array but can take less than O(n) if the timer data is represented by a more complex data structure such as a heap.


        If the system has a hardware interval timer instead of a simple clock, a program can set the interval timer to expire at a time corresponding to the software timer with the earliest expiration. There is no overhead unless a timer expires, one is started, or one is stopped. Interval timers are efficient when the timer intervals are long.




        Exercise 10.2

        Analyze the interrupt handler and the start timer function for an interval timer.


        Answer:


        The interrupt handler is the same order as the clock tick timer above. The complexity of starting the timer depends on how the timers are stored. If the timers are kept in a sorted array, the start timer function is O(n).



        The first version of the project uses an interval timer to implement multiple timers, replacing the hardware timer by a POSIX:XSI ITIMER_REAL timer. When ITIMER_REAL expires, it generates a SIGALRM signal. The SIGALRM signal handler puts an entry in an event list sorted by order of occurrence. Each entry just contains a timer number giving a timer that expired.


        Figure 10.2 shows a simple implementation of five software timers represented by the timers data structure. The individual timers (designated by [0] through [4]) are represented by long entries in the array active. An array entry of �1 represents a timer that is not active. The events array keeps a list of timers that have expired, and numevents holds the number of unhandled events. The running variable, which holds the timer number of the currently running timer, will be needed for later parts of the project.



        Figure 10.2. The timers data structure with no timers active.






        Start a timer by specifying a timer number and an interval in microseconds. Figure 10.3 shows the data structure after timer [2] is started for five seconds (5,000,000 microseconds). No timers have expired, so the event list is still empty.



        Figure 10.3. The timers data structure after timer [2] has been set for five seconds.






        Just writing the information into the active array in Figure 10.2 is not enough to implement a timer. The program must set the ITIMER_REAL timer for 5,000,000 microseconds. On delivery of a SIGALRM signal, the program must clear the active array entry and insert an entry in the events array. Figure 10.4 shows the timers data structure after ITIMER_REAL expires.



        Figure 10.4. The timers data structure after timer [2] expires.










          [ Team LiB ]



          3.12 Working with NULL Values




          I l@ve RuBoard










          3.12 Working with NULL Values




          3.12.1 Problem



          You're trying to
          compare column values to NULL, but it
          isn't working.





          3.12.2 Solution



          You have to use the proper comparison operators:
          IS NULL,
          IS NOT
          NULL, or <=>.





          3.12.3 Discussion



          Conditions involving NULL are special. You cannot
          use = NULL or
          != NULL to look for
          NULL values in columns. Such comparisons always
          fail because it's impossible to tell whether or not
          they are true. Even NULL =
          NULL fails. (Why? Because you
          can't determine whether one unknown value is the
          same as another unknown value.)



          To look for columns that are or are not NULL, use
          IS NULL or
          IS NOT NULL.
          Suppose a table taxpayer contains taxpayer names
          and ID numbers, where a NULL ID indicates that the
          value is unknown:



          mysql> SELECT * FROM taxpayer;
          +---------+--------+
          | name | id |
          +---------+--------+
          | bernina | 198-48 |
          | bertha | NULL |
          | ben | NULL |
          | bill | 475-83 |
          +---------+--------+


          You can see that = and != do
          not work with NULL values as follows:



          mysql> SELECT * FROM taxpayer WHERE id = NULL;
          Empty set (0.00 sec)
          mysql> SELECT * FROM taxpayer WHERE id != NULL;
          Empty set (0.01 sec)


          To find records where the id column is or is not
          NULL, the queries should be written like this:



          mysql> SELECT * FROM taxpayer WHERE id IS NULL;
          +--------+------+
          | name | id |
          +--------+------+
          | bertha | NULL |
          | ben | NULL |
          +--------+------+
          mysql> SELECT * FROM taxpayer WHERE id IS NOT NULL;
          +---------+--------+
          | name | id |
          +---------+--------+
          | bernina | 198-48 |
          | bill | 475-83 |
          +---------+--------+


          As of MySQL 3.23, you can also use <=> to
          compare values, which (unlike the = operator) is
          true even for two NULL values:



          mysql> SELECT NULL = NULL, NULL <=> NULL;
          +-------------+---------------+
          | NULL = NULL | NULL <=> NULL |
          +-------------+---------------+
          | NULL | 1 |
          +-------------+---------------+




          3.12.4 See Also



          NULL values also behave specially with respect to
          sorting and summary operations. See Recipe 6.6 and
          Recipe 7.9.










            I l@ve RuBoard



            Section B.5. Regular Expressions







            B.5. Regular Expressions

            When using the operators REGEXP,
            RLIKE, and NOT REGEXP, you may need
            special characters and parameters to be able to search for data based on
            regular expressions. Table B-5 lists the
            special characters, and Table B-6 shows
            special constructs that may be used. In keeping with convention, patterns
            to match are given within quotes. As an example of a regular expression
            used with a SELECT statement, suppose that we want to
            find the name of a particular student in a college's database, but we
            can't quite remember his last name. All we remember is that it's something
            like Smith, but it could be
            Smithfield or maybe Smyth. We
            could run an SQL statement like the following to get a list of
            possibilities:

            SELECT student_id, 
            CONCAT(name_first, SPACE(1), name_last) AS Student
            FROM students
            WHERE name_last REGEXP 'Smith.*|Smyth';


            As an example using a pattern-matching construct, suppose that we
            suspect there are a few student records in which the name columns contain
            numeric characters. Suppose also that there are some student records in
            which the social_security column
            contains characters other than numbers or dashes. We could search for them
            by executing an SQL statement like the following:

            SELECT student_id, soc_sec, 
            CONCAT(name_first, SPACE(1), name_last) AS Student
            FROM students
            WHERE CONCAT(name_first, name_last) REGEXP '[[:digit:]]+'
            OR soc_sec REGEXP '[[:alpha:]]+';


            As an example of a construct using a character name, suppose that
            the column containing Social Security tax identification numbers (i.e.,
            soc_sec) shouldn't contain the usual hyphen separator
            (i.e., 443-78-8391). We could enter an SQL statement
            like the following to find records with hyphens in that column:

            SELECT student_id, soc_sec, 
            CONCAT(name_first, SPACE(1), name_last) AS Student
            FROM students
            WHERE soc_sec REGEXP '[[.hyphen.]]+';


            To find any rows that do not specifically meet the format for the
            Social Security number (i.e., nnn-nn-nnnn), we
            could use this longer but more specific regular expression:

            SELECT student_id, soc_sec, 
            CONCAT(name_first, SPACE(1), name_last) AS Student
            FROM students
            WHERE soc_sec NOT REGEXP
            '[[:digit:]]{3}[[.hyphen.]]{1}[[:digit:]]{2}[[.hyphen.]]{1}[[:digit:]]{4}';


            Notice that this statement uses the curly braces after each
            construct to specify the exact number of characters or digits
            permitted.

            Table B-5. Pattern-matching characters
            CharacterUse
            ^Matches the beginning of the string.
            $Matches the beginning of the string.
            .Matches any character, space, or line
            ending.
            *Matches zero or more of the characters immediately
            preceding.
            +Matches one or more of the characters immediately
            preceding.
            ?Matches zero or one of the characters immediately
            preceding.
            |An OR operator; matches the
            characters before or after it (e.g.,
            'Russell|Rusty').
            (characters)*Matches zero or more occurrences of the sequence of
            characters given in parentheses.
            {number}Specifies the number of occurrences of the previous
            pattern given.
            {number,number}Specifies the minimum number of occurrences of the
            previous pattern given, followed by the maximum number of
            occurrences. If only the minimum number is omitted, 0 is assumed.
            If just the maximum number is omitted, unlimited is
            assumed.
            [x-x]Specifies a range of characters in alphabetical order
            (e.g., '[a-g]' for the first seven lowercase letters), or numbers
            in numeric sequence (e.g., '[0-9]' for all
            numbers).


            Table B-6. Pattern-matching constructs
            ConstructUse
            [.character.]Matches the given character or character name (e.g.,
            backslash, carriage return,
            newline,
            tab).
            [=character=]Matches characters of the same class as the character
            given.
            [[:<:]]Matches the beginning of a word.
            [[:>:]]Matches the end of a word.
            [:alnum:]Matches alphanumeric characters.
            [:alpha:]Matches alphabetical characters.
            [:blank:]Matches a blank or whitespace
            characters.
            [:cntrl:]Matches control characters.
            [:digit:]Matches digits.
            [:lower:]Matches lowercase alphabetical
            characters.
            [:print:]Matches graphic and space characters.
            [:punct:]Matches punctuation characters.
            [:space:]Matches space, carriage return, newline, and tab
            characters.
            [:upper:]Matches uppercase alphabetical
            characters.
            [:xdigit:]Matches hexadecimal characters.









            Hack&nbsp;4.&nbsp;Customize Authentication with PAMs










            Hack 4. Customize Authentication with PAMs



            Modern Linux systems use Pluggable Authentication Modules (PAMs) to provide flexible authentication for services and applications. Here are the gory details you'll need in order to use PAMs to quickly and flexibly secure your systems.


            Many Linux applications require authentication of one type or another. In days gone by, each authentication-aware application was compiled with hardwired information about the authentication mechanism used by the system on which it was running. Changing or enhancing a system's authentication mechanism therefore required all such applications to be updated and recompiled, which is tedious even when you have the source code for all of the relevant applications on your system.


            Enter PAMs, which provide a flexible and dynamic mechanism for authenticating any application or service that uses them. Applications or services compiled with the Linux-PAM library use text-format configuration files to identify their authentication requirements. Using PAMs on your system lets you modify authentication requirements or integrate new authentication mechanisms by simply adding entries to the PAM configuration file that is used by a specific application or service.


            Though the information contained here may seem like overkill at first glance, knowing about PAMs and how PAM configuration files work is necessary background for the next four hacks, which explain how to integrate specific types of modern authentication into your Linux system without rewriting or recompiling the wheel. Read on, sysadmins!



            1.5.1. PAM Overview



            PAMs are shared library modules that are automatically loaded by applications that were compiled with the primary Linux-PAM authentication library. Applications that use PAMs (or PAM modules, are they're sometimes called) are typically referred to as PAM-aware applications.


            PAMs satisfy different parts of the authentication requirements for PAM-aware applications, much like reusable code and libraries do for applications in general. For example, a PAM-aware version of the login program can invoke a variety of PAMs that check things such as whether the user logging in as root is on a terminal listed as a secure terminal, whether users are allowed to log in on the system at the moment, and other similar authentication requirements. Because PAMs are shared library modules, a PAM-aware version of the rsh program can reuse the same "are users allowed to log in on the system now?" PAM as the PAM-aware version of login, but then apply other rules that are more relevant to rsh than to login. PAM modules themselves are now typically stored in the directory /lib/security, though some older Linux distributions stored PAMs in /usr/lib/security.


            The PAMs used by different PAM-aware applications are defined in one of two ways. In modern PAM implementations, they are controlled by application-specific configuration files found in the directory /etc/pam.d. In older PAM implementations, all PAM modules used by the applications on a system were defined in a single central configuration file, /etc/pam.conf. The older approach is still supported, but it's deprecatedto maintain backward compatibility while encouraging the modern approach, the contents of the directory /etc/pam.d are used instead of the /etc/pam.conf file if both exist on your system. This hack focuses on PAM configuration files in /etc/pam.d, since that's the way PAMs are used on most modern systems.




            1.5.2. Per-Application/Service PAM Configuration Files


            Each PAM configuration file in /etc/pam.d has the same name as the PAM-aware application or service it is associated with and contains the PAM rules used during its authentication process. The name of the configuration file to use is derived from the first parameter passed to the Linux-PAM library's pam_start() function, which is the name of the service that is being authenticated (often the same as the name of the application, for convenience's sake). These files can also contain commentsany characters on a line that follow the traditional hash mark (#) are interpreted as a comment.


            Each non-comment line in one of the files in /etc/pam.d defines how a single PAM module is used as part of the authentication process for the associated application or service. Each of these files can contain four fields separated by whitespace, the first three of which are mandatory. These fields have the following meaning and content:



            module-type



            The type of PAM module defined on the line. A PAM module's type defines how it is used during the authentication process. Valid values are:




            auth



            Identifies an authentication check to verify the user's identity or that system requirements have been met. Common system requirements are that a service can be started at the current time (for example, that /etc/nologin does not exist when a user is trying to log in), that an acceptable device is being used (i.e., the device is listed in /etc/securetty), whether the user is already the root user, and so on.





            account



            Verifies whether the user can authenticate based on system requirements such as whether the user has a valid account, the maximum number of users on the system, the device being used to access the system, whether the user has access to the requested application or service, and so on.





            password



            Verifies a user's ability to update authentication mechanisms. There is usually one module of type password for each auth entry that is tied to an authentication mechanism that can be updated.





            session



            Identifies modules associated with tasks that must be done before the associated service or application is activated, or just before the termination of that service or application. Modules of this type typically perform system functions such as mounting directories, logging audit trail information, or guaranteeing that system resources are available.







            control-flag



            The implications of the return value from the specified PAM module. Valid values are:




            required



            Indicates that success of the PAM module is mandatory for the specified module type. The failure of any PAM marked as required for a specific module-type (such as all labeled as auth) is reported to the associated application or service only after all required PAMs for that module type have been executed.





            requisite



            Indicates that failure of the PAM module will immediately be reported to the associated application or service.





            sufficient



            Indicates that success of the PAM module satisfies the authentication requirements of this module type. If no previous required PAM has failed, no other PAMs for the associated module type are executed. Failure of a PAM identified as sufficient is ignored as long as subsequent required modules for that module type return success. If a previous required PAM has failed, the success of a PAM marked as sufficient is ignored.





            optional



            Indicates that success of the PAM module is not critical to the application or service unless it is the only PAM for a specified module type. If it is, its success or failure determines the success or failure of the specified module type.







            module-path



            The name of the PAM module associated with this entry. By default, PAM modules are located in /lib/security, but this field can also identify modules located in other directories by specifying the absolute path and filename of a PAM module.





            arguments



            Optional, module-specific arguments.




            Well, that was mind-numbing but necessary reference information. To see all this in action, let's look at an example.




            1.5.3. PAMs Used by the login Process




            The configuration file for the PAMs used by the login program is the file /etc/pam.d/login. On a Red Hat system of recent vintage, this file contains the following entries:



            #%PAM-1.0
            auth required pam_securetty.so
            auth required pam_stack.so service=system-auth
            auth required pam_nologin.so
            account required pam_stack.so service=system-auth
            password required pam_stack.so service=system-auth
            session required pam_stack.so service=system-auth
            session optional pam_console.so



            The first line is a comment that identifies this PAM as conforming to the PAM 1.0 specification.


            The second, third, and fourth lines define the auth (authentication) requirements for system logins, all of which must succeed because they are identified as required. The second line invokes the PAM module pam_securetty.so to check whether the user is logged in on a secure terminal, as defined in the file /etc/securetty. The third line invokes the pam_stack.so PAM module, a clever module used primarily on Red Hatinspired systems that enables you to call the entire set of PAM requirements defined for a different service or application (and thus described in a separate file by that name in /etc/pam.d). In this case it calls the set (stack) of requirements defined for the system-auth service. We'll look at that laterfor now, it's sufficient to know that the authentication requirements specified in that file must be satisfied. Finally, to wrap up the auth module-type entries for the login program, the fourth line invokes the pam_nologin.so PAM module to check whether logins are allowed on the system at the current time.


            The fifth line in this file identifies the requirements for the account module-type, which in this case uses the pam_stack.so PAM module to verify that the set of requirements for the system-auth service have been satisfied.


            Similarly, the sixth line in this file identifies the requirements for the password module-type, which also uses the pam_stack.so PAM module to verify that the set of requirements for the system-auth service have been satisfied.


            Finally, the seventh and eighth lines in this file identify session requirements for the login program. The seventh line uses the familiar pam_stack.so PAM module to verify that the set of requirements for the system-auth service were satisfied. The eighth line in this file defines an optional requirement that the user be running on the console. If this module succeeds, the user is granted any additional privileges associated with this PAM module. If this module fails, authentication succeeds as long as the previous required modules have completed successfully, but the user doesn't get the bonus privileges.


            Now let's look at the /etc/pam.d/system-auth file on the same system, which contains the following:



            #%PAM-1.0
            # This file is auto-generated.
            # User changes will be destroyed the next time authconfig is run.
            auth required /lib/security/pam_env.so
            auth sufficient /lib/security/pam_unix.so likeauth nullok
            auth required /lib/security/pam_deny.so
            account required /lib/security/pam_unix.so
            password required /lib/security/pam_cracklib.so retry=3 type=
            password sufficient /lib/security/pam_unix.so nullok use_authtok md5
            shadow
            password required /lib/security/pam_deny.so
            session required /lib/security/pam_limits.so
            session required /lib/security/pam_unix.so



            Now that you grok PAM configuration files, you can see that the auth module-type first requires that the pam_env.so module succeed, then tries the pam_unix.so module, which is a generic module that can perform auth, account, password, and session functions (depending on how it is called). When called for the auth module-type, it verifies a user's identity, sets credentials if successful, and so on. If this module succeeds, the following required entry for the pam_deny.so module isn't executed. If the pam_unix.so module fails, pam_deny.so executes, which returns a failure code to ensure that the specified module-type will fail. In our login example, where another auth request (for pam_nologin.so) follows the invocation of the contents of the system-auth PAM stack, that auth request is executed, but its value isn't important because pam_deny.so is required and has already indicated failure.


            Next, the account module-type requires that the pam_unix.so module succeedin this case, pam_unix.so provides default account checks.


            Following the account check, the first password module-type line specifies that pam_cracklib.so be used when setting passwords to select a password that isn't easily cracked, based on the contents of a database of easily cracked passwords (/usr/lib/cracklib_dict.pwd on Red Hat systems). Arguments to this module give the user three chances to select a password (by passing the argument retry=3) and specify that this password isn't for any specific type of authentication, such as LDAP or NIS (by passing a null name using the type= argument). If this module succeeds, the second password module-type line invokes the standard pam_unix.so module, with arguments specifying that null passwords are acceptable but can't be set by users (nullok), not to prompt for a password but to use any password that succeeded in a previous PAM of module-type password (use_authtok), that passwords use md5 hashing by default (md5), and that the system uses the /etc/shadow file to hold passwords (shadow). If this module fails, the user is denied access to the application or service that invoked the system-auth service through the next line, which invokes the pam_deny.so module to ensure failure of the password auth-type.


            Finally, the session checks set system limits using the pam_limits.so module, which provides functions to initiate and terminate sessions.


            If you need to take a few aspirin after parsing each entry in these files, join the club. But even though it's a pain, security is one of any sysadmin's most important responsibilities. If it's any consolation, think how complex the code to implement all of this would have been without the flexibility that PAMs provide!




            1.5.4. Configuration and More Configuration


            The text-format files in /etc/pam.d control the PAMs associated with each authentication-aware application or service. Some of these PAMs themselves use optional configuration files to further refine their behavior. The configuration files for individual PAMs are located in the directory /etc/security. Though these files must exist, they do not need to contain any useful informationthey are there in case you want to take advantage of the advanced configuration options that they provide. Here is a list of the files in this directory that are found on a variety of Linux systems:




            access.conf



            Provides fine-grained access control for logins. Used by the pam_access.so module.





            console.apps



            A directory that contains a file for each privileged application that a user can use from the console. The name of each file is the same as the base-name of the application with which it is associated. These files must exist but can be empty. When they have contents, these files typically contain environment variables associated with the applications that match their names. Used by the pam_console.so module on Red Hatinspired Linux systems.





            console.perms



            Defines the device permissions granted to privileged users when logged in on the console, and the permissions to which those devices revert when the user logs out. Used by the pam_console.so module on Red Hatinspired Linux systems.





            group.conf



            Provides per-session group membership control. Used by the pam_group.so module.





            limits.conf



            Provides a per-user mechanism for setting system resource limits. Used by the pam_limits.so module.





            pam_env.conf



            Provides a mechanism for setting environment variables to specific values. Used by the pam_env.so module.





            pam_pwcheck.conf



            Provides options for identifying the mechanism used when evaluating password strength. Used by the pam_pwcheck.so module on SUSE-inspired Linux systems.





            pam_unix2.conf



            Provides options for advanced configuration of traditional password checking. Used by the pam_unix2.so module on SUSE-inspired systems.





            time.conf



            Provides a mechanism for imposing general or user-specific time restrictions for system services and applications. Used by the pam_time.so module.






            1.5.5. What if PAM Configuration Files Are Missing?



            Applications that use PAMs are very powerful, and correct configuration is very important. However, the Linux-PAM library does provide a default configuration file for any applications and services that do not have their own configuration files. This is the file /etc/pam.d/other. Since a missing configuration file generally indicates a misconfigured system (or that someone has imported a PAM-aware binary without thinking things through), the /etc/pam.d/other file implements extremely paranoid security, as in the following example:



            #%PAM-1.0
            auth required pam_deny.so
            account required pam_deny.so
            password required pam_deny.so
            session required pam_deny.so



            In this example, any request for any module-type that falls through to this PAM configuration file will return a failure code. A slightly more useful version of this file is the following:



            #%PAM-1.0
            auth required pam_deny.so
            auth required pam_warn.so
            account required pam_deny.so
            account required pam_warn.so
            password required pam_deny.so
            password required pam_warn.so
            session required pam_deny.so
            session required pam_warn.so



            Because subsequent required entries for a given module-type are still executed, each module-type entry first executes the pam_deny.so PAM, which denies access to the requested service, and then also executes the pam_warn.so PAM, which logs a warning message to the system log.




            1.5.6. See Also


            • man pam (where pam is the name of a PAM module without the .so extension)

            • http://www.ymbnet.lkams.kernel.org/pub/linux/libs/pam/













            People













            People


            Fallacy 3


            Programming can and should be egoless.


            Discussion

            There once was the first software engineering best-selling book. It was called The Psychology of Computer Programming (Weinberg 1971). There was a peculiar idea contained among the many excellent ideas of that book. It was the idea that the task of programming should be egoless. Programmers, the author said, should not invest their ego in the product they were building. There was, of course, an excellent reason for the author to take that position. Too many programmers, this author was saying, get so ego-invested in their product that they lose all objectivity. They see error reports as personal attacks. They see review sessions as threats. They see any questioning of what they have accomplished as being counterproductive.


            What's the alternative to an ego-invested programmer? A team-player programmer. The team player sees the software product as a team effort and a team achievement. Error reports and reviews and questions become team inputs to help improve the product, not threatening attacks to derail progress.


            It's hard to argue, at first glance, with the notion of the egoless programmer. Certainly the defensive programmer, caught up in his or her ego, is not open to the changes that he or she inevitably must make to improve the product. But after further thought, this notion begins to unravel. It is all well and good to advocate egoless programming, but the fact of the matter is that human ego is a very natural thing, and it is difficult to find people who can�or even should�divorce their ego from their work. Contemplate, for example, the notion of an egoless manager. That idea, of course, is preposterous! The ego of the typical manager is the driving force that makes him or her effective. We can no more divorce ego from the average programmer, I would assert, than we can or should divorce it from the average manager. We can strive, however, to keep that ego under control.


            Controversy

            Even Jerry Weinberg, the author of The Psychology of Computer Programming (1971), has revisited the notion of egoless programming in recent years because he has realized that it is controversial. He still believes in what he was trying to say in 1971; he believes that those who oppose the notion are taking it all too literally. It is hard to disagree with the good things that can emerge from separating ego and the programmer. Team approaches, in the form of reviews and inspections and the once-lauded chief programmer teams and perhaps even Extreme Programming's pair programming, are generally seen by many as better than their alternatives. And programmers really do need to be open to critique; the fact that we cannot write error-free programs, hammered home so many times in this book, means that programmers will always have to face up to their technical foibles and frailties.


            But still, there must be some middle ground here. One of the reasons Communism eventually failed is that it assumed that we could all accept the philosophy "from each according to his ability, to each according to his need." The assumption that we could all subjugate our needs to those of others is about as faulty as the assumption that we can subjugate our egos for the betterment of the team. A system that works will have to acknowledge fundamental human traits and work within the bounds they create. And ego is one of those traits.


            Source

            Weinberg's book, listed in the following Reference section, is a classic book. Don't let my disagreement with the notion of "egoless" deter you from reading it and the many excellent subsequent books that Weinberg has written. (A Silver Anniversary edition of this book was republished by Dorset House in 1998.)


            Reference


            Weinberg, Gerald. 1971. The Psychology of Computer Programming. New York: Van Nostrand Reinhold.










              Writing Startup Scripts














              Writing Startup Scripts


              Shell scripts are combinations of shell and user commands that are executed in noninteractive mode for a wide variety of purposes. Whether you require a script that converts a set of filename extensions or need to alert the system administrator by e-mail that disk space is running low, shell scripts can be used. The commands that you place inside a shell script should normally execute in the interactive shell mode as well, making it easy it to take apart large scripts and debug them line by line in your normal login shell. In this section, we will only examine shell scripts that run under the Bourne shell— although many of the scripts will work without modification using other shells, it is always best to check the syntax chart of your own shell before attempting to run the scripts on another shell.




              Processing Shell Arguments


              A common goal of writing shell scripts is to make them as general as possible, so that they can be used with many different kinds of input. For example, in the cat examples presented earlier, we wouldn’t want to have to create an entirely new script for every file that we wanted to insert data into. Fortunately, shell scripts are able to make use of command-line parameters, which are numerically ordered arguments that are accessible from within a shell script. For example, a shell script to move files from one computer to another computer might require parameters for the source host, the destination host, and the name of the file to be moved. Obviously, we want to be able to pass these arguments to the script, rather than “hardwiring” them into the code. This is one advantage of shell scripts (and Perl programs) over compiled languages like C: scripts are easy to modify, and their operation is completely transparent to the user.


              Arguments to shell scripts can be identified by a simple scheme: the command executed is referred to with the argument $0, with the first parameter identified as $1, the second parameter identified by $2, and so on, up to a maximum of nine parameters.


              Thus, a script executed with the parameters


              display_hardware.sh cdrom scsi ide

              would refer internally to “cdrom” as $1, “scsi” as $2, and “ide” as $3. This approach would be particularly useful when calling smaller scripts from the main .profile script.


              Let’s see how arguments can be used effectively within a script to process input parameters. The first script we will create simply counts the number of lines in a file (using the wc command), specified by a single command-line argument ($1). To begin with, we create an empty script file:


              touch count_lines.sh

              Next, we set the permissions on the file to be executable:


              chmod +x count_lines.sh

              Next, we edit the file


              vi count_lines.sh

              and add the appropriate code:


              #!/bin/sh
              echo "Number of lines in file " $1
              wc –l $1

              The script will take the first command-line argument, then print the number of lines, and then exit. We run the script with this command


              ./count_lines.sh /etc/group

              which gives the following output:


              Number of lines in file /etc/group
              43

              Although the individual activity of scripts is quite variable, the procedure of creating the script file, setting its permissions, editing its contents, and executing it on the command line remains the same across scripts. Of course, you may wish to make the script only available to certain users or groups for execution—this can be enabled by using the chmod command, and explicitly adding or removing permissions when necessary.






              Testing File Properties


              One of the assumptions that we made in the previous script was that the file specified by $1 actually existed; if it didn’t exist, we obviously would not be able to count the number of lines it contained. If the script is running from the command line, we can safely debug it and interpret any error conditions that arise (such as a file not existing, or having incorrect permissions). However, if a script is intended to run as a scheduled job (using the cron or at facility), it is impossible to debug in real time. Thus, it is often useful to write scripts that can handle error conditions gracefully and intelligently, rather than leaving administrators wondering why a job didn’t produce any output when it was scheduled to run.


              The number one cause of runtime execution errors is the incorrect setting of file permissions. Although most users remember to set the executable bit on the script file itself, they often neglect to include error checking for the existence of data files that are used by the script. For example, if we want to write a script that checked the syntax of a configuration file (like the Apache configuration file, httpd.conf), we need to make sure the file actually exists before performing the check; otherwise, the script may not return an error message, and we may erroneously assume that the script file is correctly configured.


              Fortunately, Bourne shell makes it easy to test for the existence of files by using the (conveniently named) test facility. In addition to testing for file existence, files that exist can also be tested for read, write, and execute permissions prior to any read, write, or execute file access being attempted by the script. Let’s revise our previous script that counted the number of lines in a file by first verifying that the target file (specified by $1) exists, and then printing the result; otherwise, an error message will be displayed:


              #!/bin/sh
              if test -a $1
              then
              echo "Number of lines in file " $1
              wc –l $1
              else
              echo "The file" $1 "does not exist"
              fi

              When we run this command, if a file exists, it should count the number of lines in the target file as before; otherwise, an error message will be printed. If the /etc/group file did not exist, for example, we’d really want to know about it:


              ./count_lines.sh /etc/group
              The file /etc/group does not exist

              There may be some situations where we want to test another file property. For example, the /etc/shadow password database must only be readable by the superuser. Thus, if we execute a script to check whether or not the /etc/shadow file is readable by a nonprivileged user, it should not return a positive result. We can check file readability by using the -r option rather than the -a option. Here’s the revised script:


              #!/bin/sh
              if test –r $1 then
              echo "I can read the file " $1
              else
              echo "I can't read the file" $1
              fi

              The following file permissions can also be tested using the test facility:





              • -b File is a special block file.





              • -c File is a special character file.





              • -d File is a directory.





              • -f File is a normal file.





              • -h File is a symbolic link.





              • -p File is a named pipe.





              • -s File has nonzero size.





              • -w File is writable by the current user.





              • -x File is executable by the current user.








              Looping


              All programming languages have the ability to repeat blocks of code for a specified number of iterations. This makes performing repetitive actions very easy for a well-written program. The Bourne shell is no exception. It features a for loop, which repeats the actions of a code block for a specified number of iterations as defined by a set of consecutive arguments to the for command. In addition, an iterator is available within the code block to indicate which of the sequence of iterations that will be performed is currently being performed. If that sounds a little complicated, let’s have a look at a concrete example, which uses a for loop to generate a set of filenames. These filenames are then tested using the test facility to determine whether or not they exist:


              #!/bin/sh
              for i in apple orange lemon kiwi guava
              do
              DATAFILE=$i".dat"
              echo "Checking" $DATAFILE
              if test -s $DATAFILE
              then
              echo $DATAFILE "is OK"

              else
              echo $DATAFILE "has zero-length"

              fi
              done

              The for loop is repeated five times, with the variable $i taking on the values apple, orange, lemon, kiwi, and guava. Thus, when on the first iteration, when $i=apple, the shell interprets the for loop in the following way:


              FILENAME="apple.dat"
              echo "Checking apple.dat"
              if test -s apple.dat
              then
              echo "apple.dat has zero-length"
              else
              echo "apple.dat is OK"
              fi

              If we run this script in a directory with files of zero length, we would expect to see the following output:


              ./zero_length_check.sh
              Checking apple.dat
              apple.dat is zero-length
              Checking orange.dat
              orange.dat is zero-length
              Checking lemon.dat
              lemon.dat is zero-length
              Checking kiwi.dat
              kiwi.dat is zero-length
              Checking guava.dat
              guava.dat is zero-length

              However, if we entered data into each of the files, we should see them receive the “OK” message:


              ./zero_length_check.sh
              Checking apple.dat
              apple.dat is OK
              Checking orange.dat
              orange.dat is OK
              Checking lemon.dat
              lemon.dat is OK
              Checking kiwi.dat
              kiwi.dat is OK
              Checking guava.dat
              guava.dat is OK





              Using Shell Variables


              In the previous example, we assigned different values to a shell variable, which was used to generate filenames for checking. It is common to modify variables within scripts by using export, and to attach error codes to instances where variables are not defined within a script. This is particularly useful if a variable that is available within a user’s interactive shell is not available in their noninteractive shell. For example, we can create a script called show_errors.sh that returns an error message if the PATH variable is not set:


              #!/bin/sh
              echo ${PATH:?PATH_NOT_SET}

              Of course, since the PATH variable is usually set, we should see output similar to the following:



              # ./path_set.sh
              /sbin:/bin:/usr/games/bin:/usr/sbin:/root/bin:/usr/local/bin:/usr/local/sbin/
              :/usr/bin:
              /usr/X11R6/bin: /usr/games:/opt/gnome/bin:/opt/kde/bin


              However, if the PATH was not set, we would see the following error message:


              ./show_errors.sh: PATH_NOT_SET

              It is also possible to use system-supplied error messages, by not specifying the optional error string:


              #!/bin/sh
              echo ${PATH:?}

              Thus, if the PATH variable is not set, we would see the following error message:


              # ./path_set.sh
              ./showargs: PATH: parameter null or not set

              We can also use the numbered shell variables ($1, $2, $3, and so forth) to capture the space-delimited output of certain commands, and perform actions based on the value of these variables using the set command. For example, the command


              # set `ls`

              will sequentially assign each of the fields within the returned directory listing to a numbered shell variable. So, if our directory listing contained the entries


              apple.dat   guava.dat   kiwi.dat   lemon.dat   orange.dat

              We could retrieve the values of these filenames by using the echo command:


              # echo $1
              apple.dat
              # echo $2
              guava.dat
              # echo $3
              kiwi.dat
              # echo $4
              lemon.dat
              # echo $5
              orange.dat

              This approach is very useful if your script needs to perform some action based on only one component of the date. For example, if you wanted to create a unique filename to assign to a compressed file, you could combine the values of each variable with a .Z extension to produce a set of strings like orange.dat.Z.













              Section 40.1.&nbsp; Symmetries: Operations That Cancel Each Other









              40.1. Symmetries: Operations That Cancel Each Other


              Many business transactions have natural counterparts, as there is a kind of resource-balancing process going on. Goods are traded for cash, currencies are traded based on an exchange rate, time is exchanged for consulting fees, and bookings should all expire and bought goods replaced or returned.


              With hiring in the RPS system, there is a balancing between the rental items going out and coming back or, occasionally, being exchanged for cash or written off because they weren't returned. So it's easy to write a program with a model for that process, involving multiple clients and rentals. Interleaving the rentals and returns is more likely to stretch the system and find errors, so we'll do that.


              The design of model-based test generation entails several issues about how we generate the tests, run and report results, and develop the underlying model. One approach would be to generate Fit tables in files and run Fit on them as a group. But a more direct way is to generate the tables and run them with CustomRunner.


              We don't want to have to manually run the tests and check the results afterward. It will be convenient to have the tests run automatically, maybe all the time. We want to be informed only when things go wrong, with sufficient information to be able to replicate any errors that are found.


              Finally, we want to keep the model as simple as possible, for two reasons. One is to reduce the cost of creating it. The second is to reduce the chance of finding errors in the model rather than in our system.









                6.7 Exercise: Audio



                [ Team LiB ]






                6.7 Exercise: Audio


                The exercises in this section assume that the operating system handles the audio device in a way similar to how the Solaris operating environment handles it.











                1. Add the following access functions to the audio object of Program 6.16.


                  1. The play_file function plays an audio file. It has the following prototype.


                    int play_file(char *filename);

                    The play_file outputs the audio file specified by filename to the audio device, assuming that the speaker has already been opened. If successful, play_file returns the total number of bytes output. If unsuccessful, play_file returns �1 and sets errno.

                  2. The record_file function saves incoming audio data to a disk file. It has the following prototype.


                    int record_file(char *filename, int seconds);

                    The record_file function saves audio information for a time interval of seconds in the file given by filename, assuming that the microphone has already been opened. If successful, record_file returns the total number of bytes recorded. If unsuccessful, record_file returns �1 and sets errno.

                  3. The get_record_sample_rate function determines the sampling rate for recording. It has the following prototype.


                    int get_record_sample_rate(void);

                    If successful, get_record_sample_rate returns the sampling rate for recording. If unsuccessful, get_record_sample_rate returns �1 and sets errno.

                  4. The get_play_buffer_size returns the buffer size that the audio device driver uses to transfer information to the audio output device. It has the following prototype.


                    int get_play_buffer_size(void);

                    If successful, get_play_buffer_size returns the buffer size for recording. If unsuccessful, get_play_buffer_size returns �1 and sets errno.

                  5. The get_play_sample_rate function determines the sampling rate for playing. It has the following prototype.


                    int get_play_sample_rate(void);

                    If successful, get_play_sample_rate returns the sampling rate used for playing audio files on the speaker. If unsuccessful, get_play_sample_rate returns �1 and sets errno. A rate of 8000 samples/second is considered voice quality.

                  6. The set_play_volume function changes the volume at which sound plays on the speaker. It has the following prototype.


                    int set_play_volume(double volume);

                    The set_play_volume sets the gain on the speaker. The volume must be between 0.0 and 1.0. If successful, set_play_volume returns 0. If unsuccessful, set_play_volume returns �1 and sets errno.

                  7. The set_record_volume function changes the volume of incoming sound from the microphone. It has the following prototype.


                    int set_record_volume(double volume);

                    The set_record_volume function sets the gain on the microphone. The volume value must be between 0.0 and 1.0. If successful, set_record_volume returns 0. If unsuccessful, it returns �1 and sets errno.


                2. Rewrite Program 6.17 to copy from the microphone to the speaker, using the preferred buffer size of each of these devices. Call get_record_buffer_size and get_play_buffer_size to determine the respective sizes. Do not assume that they are the same in your implementation.

                3. Use the record_file function to create eight audio files, each of which is ten seconds in duration: pid1.au, pid2.au, and so on. In the file pid1.au, record the following message (in your voice): "I am process 1 sending to standard error". Record similar messages in the remaining files. Play the files back by using the play_file function.

                4. Be sure to create a header file (say, audiolib.h) with the prototypes of the functions in the audio library. Include this header file in any program that calls functions from this library.

                5. Record your speaking of the individual numerical digits (from 0 to 9) in ten different files. Write a function called speak_number that takes a string representing an integer and speaks the number corresponding to the string by calling play_file to play the files for the individual digits. (How does the program sound compared to the computer-generated messages of the phone company?)

                6. Replace the fprintf statement that outputs the various IDs in Program 3.1 on page 67 with a call to play_file. For the process with i having value 1, play the file pid1.au, and so on. Listen to the results for different numbers of processes when the speaker is opened before the fork loop. What happens when the speaker is opened after the fork? Be sure to use snprintf to construct the filenames from the i value. Do not hardcode the filenames into the program.

                7. Make a recording of the following statement in file pid.au: "My process ID is". Instead of having each process in the previous part play a pidi.au file corresponding to its i number, use speak_number to speak the process ID. Handle the parent and child IDs similarly.

                8. Redesign the audio object representation and access functions so that processes have the option of opening separately for read and for write. Replace audio_fd with the descriptors play_fd and record_fd. Change the open_audio so that it sets both play_fd and record_fd to the file descriptor value returned by open. Add the following access functions to the audio object of Program 6.16.


                  1. The open_audio_for_record function opens the audio device for read (O_RDONLY). It has the following prototype.


                    int open_audio_for_record(void);

                    The function returns 0 if successful or �1 if an error occurs.

                  2. The open_audio_for_play function opens the audio device for write (O_WRONLY). It has the following prototype.


                    int open_audio_for_play(void);

                    The open_audio_for_play function returns 0 if successful or �1 if an error occurs.







                  [ Team LiB ]