/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * @file MyroC-display.c * @brief Implementation file for downloading an image * from the Scribbler 2 camera to the local workstation * * this version uses glutHideWindow when image windows time out * gluttHideWindow seems to work fine on Mac OS X and on some Linux machines * unfortunately, when run on amd graphics cards, glutHideWindow causes * window information to be lost, and subsequent calls to glutHowWindow fail * thus, this version of MyroC-display.c works fine on Mac OS X and * but not on Linux machines with various graphics cards * * Revision History * * Version 1.0 based on a C++ package by April O'Neill, David Cowden, * Dilan Ustek, Erik Opavsky, and Henry M. Walker * * Developers of the C package for Linux: * Creators Version 2.0 (C functions for utilities,general,sensors,movement): * Spencer Liberto * Dilan Ustek * Jordan Yuan * Henry M. Walker * Contributors Version 2.2-2.3: (C functions for image processing) * Anita DeWitt * Jason Liu * Nick Knoebber * Vasilisa Bashlovkina * Revision for Version 2.4: (image row/column made to match matrix notation) * Henry M. Walker * * Revisions for Version 3.0 * Henry M. Walker * * C ported to Macintosh * Linux/Mac differences required for connections — otherwise same code * OpenGL used to display images, replacing ImageMagick * same [new] code used for both Linux and Macintosh * 1 process for robot control * 1 process needed for each titled window (not each image, as in 2.2-2.4) * Blocking options (negative duration parameter) * utilize separate thread timer * MyroC implementation files organized by user function as follows: * * Revisions for Version 3.1 * Henry M. Walker * * Picture struct and image functions revised to allow * 192 by 256 images from origial Fluke camera * 266 x 427 low-resolution images from Fluke 2 * (high-resolution (800 x 1280) too large for more than 2-4 * variables on run-time stack) * storage , retrieval, and display of any images up to 266 x 427 * * Revisions for Version 3.2 * Henry M. Walker * * Practical range of rBeep duration identified as <= 3.0014 seconds * Image display and processing resolves several matters and adds functionality * function rDisplayPicture completely rewritten * over time OpenGL rountines had encountered troubles * on Linux, specifically glutHideWindow() hid images, but could not be * restored, with difficulties depending on the graphics card * on Mac, High Sierra generated compile warnings and * restricted threads that could display images * function rDisplayImageProcessingErrors added * function rWaitTimedImageDisplay updated substantially * * This program and all MyroC software is licensed under the Creative Commons * Attribution-Noncommercial-Share Alike 3.0 United States License. * Details available at http://creativecommons.org/licenses/by-nc-sa/3.0/us/ * ********************** implementation overview ************************* > indicates functions implemented in this file 0. LOW-LEVEL UTILITY (prototypes in MyroC-utilities.h implemented in MyroC-utilities.c) rSend rReceive rSetOpeningExchange rSetBluetoothEcho rSetEchoMode rCheckHardwareVersionSetCameraSize 1. GENERAL (mostly MyroC-general.c) 2. SENSOR (MyroC-sensors.c) rConnect (* MyroC-connect.c) a.Scribbler Sensors rDisconnect (* MyroC-connect.c) rGetLightsAll rSetConnection rGetLightTxt rFinishProcessing rGetIRAll rSetVolume rGetIRTxt rBeep rGetLine Beep2 rSetName b.Fluke Sensors rGetName rGetObstacleAll rSetForwardness rGetObstacleTxt rGetForwardness rGetBrightAll rSetLEDFront rGetBrightTxt rSetLEDBack rSetIRPower rGetBattery rGetStall Note: * MyroC-connect requires special knowledge of a Mac or Linux environment, yielding two versions: MyroC-connect-mac.c MyroC-connect-linux.c 3. MOVEMENT (MyroC-movement.c) 4. PICTURES (files as indicated) rTurnLeft rGetCameraSize (MyroC-camera.c) rTurnRight rTakePicture (MyroC-camera.c) rTurnSpeed > rDisplayPicture (MyroC-display.c) rForward > rDisplayImageProcessingErrors (MyroC-display.c) rFastForward > rWaitTimedImageDisplay(MyroC-display.c) rBackward rSavePicture (MyroC-image-file.c) rMotors rLoadPicture (MyroC-image-file.c) rStop rHardStop ************************************************************************ */ /* Overall processing constraints and plan Constraints Both OpenGL and ImageMagick on Linux and Mac OS X require a window to be controled as the primary thread of a process. Since a user may specify multiple windows (with different titles), the MyroC process must create a different process for each image window. OpenGL allows an window to be updated with a new image Thus, * the main MyroC process must be able to send a new message to the processing controling a window. * a pipe is needed from the main MyroC proces to a window process * once a window process displays an image the process must await a 'D' message to display a new image The GLUT package within OpenGL contains procedure glutHideWindow This works fine on Mac OS X This hides a window without trouble on Linux, but the corresponding glutPostRedisplay does not make a new window visible Thus, * MyroC will have to kill a process when a window is to be hidden, and a new window process created when a new image will be displayed at the same location with the same title. A window may disappear through either of two processes * If the window display is blocking, the window (and corresponding process) must go away when the duration interval is satisfied. * If the window display is non-blocking, * a timer thread can keep track of when the window should disappear * another user request may cause a new picture to be displayed, based on a new timing interval. */ /* Approach to the display of camera images * * approach outline based on two processes: * this [user] [main] program runs in one program * user/main program provides overall control of the robot, images, etc. * a child process provides OpenGL control for the display of images * * overall algorithm * utilize shared Picture struct variable * main process * provides general user interface * call to rDisplayPicture * if second/child process not running, * create pipe to send Picture images from main to child process * create child process * initialize graphics environment * establish shared Picture image as texture for image * establish new_image procedure as glutIdleFunc * if second process already running * first thread sends image through pipe to child process * redisplay process in child process * reads image * uses glTexSubImage2D to establish shared Picture image * uses glutPostRedisplay to force display of new image * image process (one process for each window) * create/initialize new image: procedure init_opengl_window * refresh image (required by openGL): procdure display * update image: procedure update_display * * synchronization protocol: all record keeping performed in main process * for new or updated image: * main process sends data to image process through socket * image process sends confirmation when done through socket */ /* OpenGL frameworks */ #ifdef __APPLE__ /* OpenGL headers for Mac OS X environment */ #include #else /* OpenGL headers for Linux environment */ #include #endif /* headers for any environment */ #include "MyroC.h" #include "MyroC-utilities.h" #include #include /* added to example for experimentation */ #include /* for strerror function */ #include /* for byte-level read/write */ #include #include /* mechanism for inter-thread communication */ #include /* needed to check if window process still exists */ #include /* for checking runtime error codes */ #include /* for killing display window, if necessary */ #include /* for determining if window process is running */ #include /* for threads, used in timing non-blocking images */ /* ********************** Declarations for MyroC internals **************** */ /* Constants for Imaage Scaling use 1 for low-resolution screens, where points in a picture same as pixels on the screen use 2 for high0resolution screens, where each pixel in the picture extends over 2 pixels on the screen */ #ifdef __APPLE__ /* With today's screens, defaulty factor is 2 */ #define WINDOW_HORIZONTAL_SCALING_FACTOR 2 #define WINDOW_VERTICAL_SCALING_FACTOR 2 #else /* on Linux, default 1 seems safest */ #define WINDOW_HORIZONTAL_SCALING_FACTOR 1 #define WINDOW_VERTICAL_SCALING_FACTOR 1 #endif /** Struct for recording details of openGL process */ typedef struct { char * window_title; // pointer to title string (unique for window) pid_t display_pid; // ID of openGL process for window int socket_id[2]; // pipe from main process to openGL process pthread_mutex_t lock; // mutex to prevent concurrent socket communication GLuint texture_objs [1]; // array for texture 1 object for openGL process int height; // height of image object (192 for original fluke) int width; // width of image object (256 for original fluke) int process_status; // current status of a display window process // values // 0: image process inactive, likely killed // 1: process active and timed to display of current image // 2: process active for indefinite time for display of current image int image_num; // sequence number of current/most recent image Pixel * pixs; // pointer to height*width 1D array of Pixels } GLimageset; /** * Struct for timing display of image */ typedef struct { double delay_time; int GLimageset_index; int image_seq_number; int thread_seq_number; } image_timing_data; /** * Struct for acting timing-thread information */ typedef struct { int thread_seq_number; pthread_t pthread; } timing_thread_info; /* ********** procedures needed in window process for openGL ********** */ /* initialize graphics environment */ void init_opengl_window (); //image process /* callback function called to redraw the scene/image being created */ void display (void); //image process /* callback function for use when CPU idle * used to handle processing other than window-based events * possible window-display updates: * either hiding current window * or preparing new image for display * receives input through pipe for displaying/updating/hiding images * format for data from pipe * byte 1: D display new image; * bytes 2-5: height of image * bytes 6-9: width of image * bytes 10-13: image sequence number * subsequent bytes: RGB pixel values as 1D array * byte 1: F current image display finished * bytes 2-5: image sequence number * */ void update_display (void); //image process /* ************* MyroC helper procedures ********************************* */ /** * take action to display image for the proper length of time */ void process_image_duration (double duration, int window_index); /** * timer to complete given openGL image after specified time */ void * display_timer_thread (void * delay_data); /* * @brief Create a new window to display a picture * @param pic RGB picture struct from Scribbler 2 camera * @param duration if duration > 0, operation is blocking * if duration <= 0, operation is non-blocking * for duration != 0, picture displayed for abs(duration) * seconds or until picture closed manually * if duration == 0, picture displayed until closed manually * @param window_index Index of existing window in image_array * white spaces will be replaced with underscores. * @param attempt_num count of how many attempts allowed before * giving up in creating new window * ( start >= 2, in case old window is being * killed when processing starts ) * @pre image_array[window_index] does not correspond * to an active window process */ void rDisplayPictureNewWindow (Picture * pic, double duration, const char * windowTitle, int window_index, int attempt_num); /* * @brief Update a picture in an existing window * @param pic RGB picture struct from Scribbler 2 camera * @param duration if duration > 0, operation is blocking * if duration <= 0, operation is non-blocking * for duration != 0, picture displayed for abs(duration) * seconds or until picture closed manually * if duration == 0, picture displayed until closed manually * @param window_index Index of existing window in image_array * white spaces will be replaced with underscores. * @pre 0 <= window_index < num_windows * image_array[window_index] corresponds to an active * window process */ void rDisplayPictureUpdate (Picture * pic, double duration, int window_index); /** * add thread sequence number, id, etc. to thread_timer array; */ void recordTimerThread (int thread_seq_num, pthread_t thread); /** * remove thread sequence number, id, etc. to thread_timer array; */ void deleteTimerThread (int thread_seq_num); /* ************* MyroC procedures **************************************** */ /** * @brief Display a picture in a new window * @param pic pointer to an RGB picture struct from Scribbler 2 camera * @param duration if duration > 0, operation is blocking \n * if duration <= 0, operation is non-blocking \n * for duration != 0, picture displayed for abs(duration) \n * seconds or until picture closed manually or * until the program terminates \n * if duration == 0, picture displayed until closed manually * @param windowTitle The title of the window that appears. * white spaces will be replaced with underscores. * @pre windowTitle is less than 100 characters. * @post image is displayed for the duration specified, \n * EXCEPT all display windows are closed * when the main program terminates. * @warning If images are displayed with a non-blocking option, and * if the user wants images to appear for a full duration, * use rWaitTimedImageDisplay or rFinishProcessing to block * processing until all image timers are finished. * Otherwise, program termination may close windows prematurely. */ void rDisplayPicture(Picture * pic, double duration, const char * windowTitle); /** @brief Turn on and off the printing of error messasge for image processing * @note Some operating systems (e.g., Mac OX S High Sierra) display * an error message whenevern a new process for image display starts * @post This function toggles this printing on and off * Initially, printing is off */ void rDisplayImageProcessingErrors (); /** * @brief Wait until all timed, non-blocking image window timers are complete * @post wait until all timed [non-blocking] images have closed \n * robot motion is unaffected by this function * * @warning images opened with duration 0.0 do not close until * images are updated or until the program terminates \n * all other image windows are closed by this function */ void rWaitTimedImageDisplay (); /* ********************** global variables ***************************** */ /* maximum number of openGL windows allowed */ #define max_windows 9 /* number of openGL windows currently defined */ int num_windows = 0; /* index of current window being processed*/ /* global variable needed, so value can be accessed by all window procedures */ int window_index; /* more recent image sequence number */ int image_seq_number = 0; /* array of data for openGL processes */ GLimageset image_array[max_windows] = {{NULL}}; /* * elements for keeping track of active threads */ int max_size_thread_array = 0; // maximum size of the current thread array int current_size_thread_array = 0; // number of elements actually containing thread info int current_thread_seq_num = 0; // a sequence number for identifying each thread int initial_thread_array_size = 10;// size of thread array when first initialized timing_thread_info * thread_info = NULL; // can't use malloc to initialize, so // actual initialization in first call to rDisplayPictureNewWindow /* *************************** Implementations **************************** */ /* **** Implementations: Image window functions ************************** */ /* initialize graphics environment */ void init_opengl_window () //image process { /* In graphics thread: * Declare initial window size, position, and display mode * (single buffer and RGBA). Open window with "hello" * in its title bar. Call initialization routines. * Register callback function to display graphics. * Enter main loop and process events. */ // printf(" image process: starting creation of image window %s\n", // image_array[window_index].window_title); /* set up command-line environment with no parameters */ int argc = 1; char * argv [1] = {"draw-texture"}; /* window initialization */ glutInit (&argc, argv); glutInitDisplayMode (GLUT_SINGLE | GLUT_RGB | GLUT_DEPTH); /* window initialization automatically defaults to pixels * non-pixel parameters not available * * parameters based on X, Y coordinates, not matrices */ glutInitWindowSize (image_array[window_index].width, image_array[window_index].height); /* position of window ultimately depends on operating system; * paramters here are suggestions to OS only * suggestions identify a grid of windows, 3-wide, * starting at the top of the monitor */ glutInitWindowPosition (470*((window_index)%3), 300*((window_index)/3)); /* display picture */ glutCreateWindow (image_array[window_index].window_title); /* select clearing (background) color * only floating-point parameters allowed */ glClearColor (0.0, 0.0, 0.0, 0.0); /* rasterize pixels directly, * rather than using interpolation between vertices of polygons */ glShadeModel (GL_FLAT); /* enable textures as background */ glEnable (GL_DEPTH_TEST); /* set pixel storage to byte alignment */ glPixelStorei (GL_UNPACK_ALIGNMENT, 1); /* name and create a texture object * this is accomplished by generating an int (actually GLubyte) index */ glGenTextures (1, image_array[window_index].texture_objs); /* the texture object will be a 2D array */ glBindTexture (GL_TEXTURE_2D, image_array[window_index].texture_objs[0]); /* specify that the texture will fill the image * other options: GL_REPEAT --- copy image as many times as needed * GL_CLAMP --- copy last pixel to end of image */ glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); /* establish initial Picture as texture for image */ glTexImage2D (GL_TEXTURE_2D, 0, GL_RGB, image_array[window_index].width, image_array[window_index].height, 0, GL_RGB, GL_UNSIGNED_BYTE, (GLubyte *) image_array[window_index].pixs); /* set up callbacks */ glutDisplayFunc (display); glutIdleFunc (update_display); /* report to parent/scribbler process that initialization mostly done */ char ch = 'I'; // printf (" image process: reporting window %s mostly set up\n", // image_array[window_index].window_title); write (image_array[window_index].socket_id[0], &ch, 1); /* start graphics event handler */ glutMainLoop (); } // init_opengl_window /* callback function called to redraw the scene/image being created */ void display (void) { /* clear all pixels */ glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); glEnable (GL_TEXTURE_2D); glTexEnvf (GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE); glBindTexture (GL_TEXTURE_2D, image_array[window_index].texture_objs[0]); /* initialize viewing values */ glMatrixMode (GL_PROJECTION); glLoadIdentity (); /* set coordinates to pixel numbers corresponding to * the size of the image */ gluOrtho2D (0, image_array[window_index].width, 0, image_array[window_index].height); /* invert the y axis, down is positive */ glScalef (1, -1, 1); /* move the origin from bottom left corner to the upper left corner */ glTranslatef (0, -image_array[window_index].height, 0); glMatrixMode (GL_MODELVIEW); /* scaling pixels versus points for image constants defined near start of this file */ glViewport (0, 0, WINDOW_HORIZONTAL_SCALING_FACTOR * image_array[window_index].width, WINDOW_VERTICAL_SCALING_FACTOR * image_array[window_index].height); /* map image to four corners of display area * note the order of the coordinates must describe successive * vertices of the surrounding window */ glBegin (GL_POLYGON); glTexCoord2f( 0.0, 0.0); glVertex2i ( 0.0, 0.0); glTexCoord2f( 1.0, 0.0); glVertex2i (image_array[window_index].width, 0); glTexCoord2f( 1.0, 1.0); glVertex2i (image_array[window_index].width, image_array[window_index].height); glTexCoord2f( 0.0, 1.0); glVertex2i ( 0.0, image_array[window_index].height); glEnd (); /* * start procesing buffered OpenGL routines */ glFlush (); glDisable (GL_TEXTURE_2D); } // display /* callback function for use when CPU idle * used to handle processing other than window-based events * possible window-display updates: * either killing current window * or preparing new image for display * receives input through pipe for displaying/updating/hiding images * format for data from pipe * byte 1: D display new image; * bytes 2-5: height of image * bytes 6-9: width of image * bytes 10-13: image sequence number * subsequent bytes: RGB pixel values as 1D array * byte 1: 0 close current window/exit window process * */ void update_display (void) { /* window update not in progress */ /* read update request from pipe */ char command = 0; // set to 0 to anticipate handling process termination /* openGL process reads/writes from 0 end of socket */ recv (image_array[window_index].socket_id[0], &command, 1, MSG_WAITALL); // printf (" image process: update_display: received %c\n", command); switch (command) { case 'D': /* prepare next image for display */ { /* need local block to declare height, width, seq_num */ int height, width; recv (image_array[window_index].socket_id[0], &height, 4, MSG_WAITALL); recv (image_array[window_index].socket_id[0], &width, 4, MSG_WAITALL); /* check window dimensions have not changed */ if ((height != image_array[window_index].height) || (width != image_array[window_index].width)) { printf ("once window opened for a title, cannot change size\n"); return; } /* recv pixel data from pipe to get picture information*/ int j; for (j = 0; j < image_array[window_index].height * image_array[window_index].width; j++) { recv (image_array[window_index].socket_id[0], &(image_array[window_index].pixs[j].R), 1, MSG_WAITALL); recv (image_array[window_index].socket_id[0], &(image_array[window_index].pixs[j].G), 1, MSG_WAITALL); recv (image_array[window_index].socket_id[0], &(image_array[window_index].pixs[j].B), 1, MSG_WAITALL); } /* be certain window will be visible */ glutShowWindow (); /* try to display next picture */ /* update existing texture with new Picture */ glTexSubImage2D (GL_TEXTURE_2D, 0, 0, 0, image_array[window_index].width, image_array[window_index].height, GL_RGB, GL_UNSIGNED_BYTE, (GLubyte *) image_array[window_index].pixs); // printf (" image process: calling glutPostRedisplay\n"); /* report to parent/scribbler process that new image mostly done */ char ch = 'U'; // for "Updated" // printf (" image process: reporting window %s mostly set up\n", // image_array[window_index].window_title); write (image_array[window_index].socket_id[0], &ch, 1); /* start graphics event handler */ glutPostRedisplay (); } break; // end phase 1 of display command; image completed == 1 to be processed yet case 0: /* null character received, perhaps due to closing of socket at end of processing exit image-window process */ // printf (" main process: closing image window: %s\n", // image_array[window_index].window_title); printf (" clean up: closing image window: %s\n", image_array[window_index].window_title); exit (0); break; default: printf ("error (char %c/int %d) in handling image window update\n", command, command); exit (1); } } // update_display /* **** Implementations: MyroC helper functions ************************** */ /** * take action to display image for the proper length of time */ void process_image_duration (double duration, int window_index) { /* if duration > 0, * block further execution for given time */ // printf (" main process: duration processing: %10.8lf\n", duration); if (duration > 0) { // printf ("duration > 0\n"); int utime = (int)(duration * 1000000); usleep (utime); /* kill process controling outdated window */ kill (image_array[window_index].display_pid, SIGKILL); waitpid (image_array[window_index].display_pid, NULL, 0); /* update status of window process --- no longer running */ image_array[window_index].process_status = 0; /* close socket for window process */ close (image_array[window_index].socket_id[1]); } /* if duration < 0, * set timer in separate thread to complete window after designated time */ else if (duration < 0) { // printf ("duration < 0\n"); // create thread, packaging needed data into struct for timer image_timing_data * delay_data = malloc (sizeof (image_timing_data)); delay_data->GLimageset_index = window_index; delay_data->delay_time = -duration; delay_data->image_seq_number = image_array[window_index].image_num; current_thread_seq_num++; delay_data->thread_seq_number = current_thread_seq_num; pthread_t thread; int err = pthread_create (&thread, NULL, &display_timer_thread, delay_data); if (err != 0) { printf ("\ncannot create timer for non-blocking image;"); printf ("error code: %s\n", strerror(err)); } /* record thread id for the window display timer thread */ recordTimerThread (current_thread_seq_num, thread); /* set thread, so that it will exit cleanly when completed */ /* if this call omitted, this process will become a "zombie thread", until main explicitly calls pthread_join */ pthread_detach (thread); } /* duration == 0: no processing needed */ else { /* image to remain displayed indefinitely; no window display timer needed */ // printf ("duration == 0\n"); image_array[window_index].process_status = 2; } } // process_image_duration /** * timer to complete given openGL image after specified time */ void * display_timer_thread (void * delay_data) { //printf (" main process: timer thread started\n"); /* make local copy of the delay data, so that subsequent processing does not * impact this timer */ image_timing_data local_delay_data = *((image_timing_data *) delay_data); int window_index = local_delay_data.GLimageset_index; int image_seq_number = local_delay_data.image_seq_number; int thread_seq_number = local_delay_data.thread_seq_number; /* with relevant data stored, delete copy in dynamic memory */ free (delay_data); /* time specified interval of delay */ int utime = (int)(1000000 * local_delay_data.delay_time); usleep (utime); /* if timer corresponds to current image being displayed, kill window */ if (image_array[window_index].image_num == image_seq_number) { /* kill process controling abandoned window */ kill (image_array[window_index].display_pid, SIGKILL); waitpid (image_array[window_index].display_pid, NULL, 0); /* update status of window process --- no longer running */ image_array[window_index].process_status = 0; /* close socket for window process */ close (image_array[window_index].socket_id[1]); // printf (" main process: process %d for window_index %d killed (*)\n\n", // image_array[window_index].display_pid, window_index); } // printf (" main process: timer completion done for sequence number: %d\n", // image_seq_number); deleteTimerThread (thread_seq_number); /* pthread procedure must return void * data */ return (void *) 0; } /* * @brief Create a new window to display a picture * @param pic RGB picture struct from Scribbler 2 camera * @param duration if duration > 0, operation is blocking * if duration <= 0, operation is non-blocking * for duration != 0, picture displayed for abs(duration) * seconds or until picture closed manually * if duration == 0, picture displayed until closed manually * @param window_index Index of existing window in image_array * white spaces will be replaced with underscores. * @param attempt_num count of how many attempts allowed before * giving up in creating new window * ( start >= 2, in case old window is being * killed when processing starts ) * @pre image_array[window_index] does not correspond * to an active window process */ void rDisplayPictureNewWindow (Picture * pic, double duration, const char * windowTitle, int window_index, int attempt_num) { /* if all attempts have been exhausted to create window process, report failure and quit */ // printf (" New Window--num_window: %d, window_index: %d, attempt_num: %d\n", // num_windows, window_index, attempt_num); if (attempt_num == 0) { printf ("attempt failed to create window %s for new picture\n", windowTitle); return; } /* create thread_array, when functionfirst called */ if (thread_info == NULL) { thread_info = (timing_thread_info*) malloc(initial_thread_array_size*sizeof (timing_thread_info)); max_size_thread_array = initial_thread_array_size; } /* create new openGL window process and prepare for image processing */ //printf ("creating new image window %s\n", windowTitle); /* set up pipe for communication to openGL process image_array[i].socket_id[0] will be I/O for openGL image process end image_array[i].socket_id[1] will be I/O for main process */ if (socketpair (AF_UNIX, SOCK_STREAM, 0, image_array[window_index].socket_id) < 0) { perror ("pipe error"); exit (1); } /* when preparation done, image will be visible/active */ image_array[window_index].process_status = 1; /* copy of Pixel array for display */ int num_pixels = pic->height * pic->width; image_array[window_index].pixs = (Pixel *) malloc (num_pixels * sizeof (Pixel)); int j; for (j = 0; j < num_pixels; j++) { image_array[window_index].pixs[j] = pic->pix_array[j / pic->width][j % pic->width]; } /* spawn child process to handle openGL image display */ if ((image_array[window_index].display_pid = fork()) < 0) { perror ("error in fork:"); printf (" error number: %d\n", errno); exit (1); } if (image_array[window_index].display_pid == 0) { /* child process handless openGL image window */ // printf (" image process: created with window index: %d, title: %s\n", // window_index, windowTitle); /* suppress error messages, unless requested */ /* bind stderr to /dev/null, so system messages for opengl do not muddy regular output */ /* reference: https://stackoverflow.com/questions/998162/is-it-possible-to-disable-stderr-in-c */ if (!displayImageErrors) { freopen("/dev/null", "w", stderr); } /* close scribbler end of socket */ close (image_array[window_index].socket_id[1]); init_opengl_window (); } else { /* parent process continues Scribbler procssing */ /* close openGL end of socket */ close (image_array[window_index].socket_id[0]); // printf (" main process: continues: window: %d, processID: %d\n", // window_index, image_array[window_index].display_pid); /* block until image window is largely set up */ errno = 0; char ch; int num_received = recv (image_array[window_index].socket_id[1], &ch, 1, MSG_WAITALL); // printf (" main process: num__received: %d, ch: %c\n", num_received, ch); /* check if process timed out before updating new window */ if (num_received != 1) { /* if process quit, try again */ rDisplayPictureNewWindow (pic, duration, image_array[window_index].window_title, window_index, attempt_num-1); } else { /* mark status of display window as having a display and visible image (subject to functioning of glutMainLoop) */ image_array[window_index].process_status = 1; /* record new sequence number for image */ image_seq_number++; image_array[window_index].image_num = image_seq_number; /* after image displayed, this copy of the pixel array no longer needed */ free (image_array[window_index].pixs); /* handle duration parameter for this image */ process_image_duration (duration, window_index); } /* image processing completed; continue with main program */ // printf (" main process: New Window thread completed\n"); } } /* * @brief Update a picture in an existing window * @param pic RGB picture struct from Scribbler 2 camera * @param duration if duration > 0, operation is blocking * if duration <= 0, operation is non-blocking * for duration != 0, picture displayed for abs(duration) * seconds or until picture closed manually * if duration == 0, picture displayed until closed manually * @param window_index Index of existing window in image_array * white spaces will be replaced with underscores. * @pre 0 <= window_index < num_windows * image_array[window_index] corresponds to an active * window process */ void rDisplayPictureUpdate (Picture * pic, double duration, int window_index) { /* send new picture to existing openGL process */ /* write display option, * height and width, * image sequence number * pixels */ //printf ("updating existing image window %d: %s\n", // window_index, // image_array[window_index].window_title); char ch = 'D'; // use char variable as write requires address /* use lock, so transmission of image data not interrupted by separate timer */ /* tell signal handler to ignore broken pipe signal for now */ struct sigaction new_actn, old_actn; new_actn.sa_handler = SIG_IGN; sigemptyset (&new_actn.sa_mask); new_actn.sa_flags = 0; sigaction (SIGPIPE, &new_actn, &old_actn); int ret_value = write (image_array[window_index].socket_id[1], &ch, 1); //printf (" write return value: %d\n", ret_value); /* restore old signal handler for pipe */ sigaction (SIGPIPE, &old_actn, NULL); /* if check of old process does not uncover defunct process or non-working socket, switch modes to create a new window process */ if (ret_value == -1) { /* if at first you don't succeed, try again */ //pthread_mutex_unlock (&image_array[window_index].lock); rDisplayPicture (pic, duration, image_array[window_index].window_title); return; } pthread_mutex_lock (&image_array[window_index].lock); write (image_array[window_index].socket_id[1], &(image_array[window_index].height), 4); write (image_array[window_index].socket_id[1], &(image_array[window_index].width) , 4); /* copy 2D Pixel array in Picture to 1D Pixel array for OpenGL and send data to OpenGL process for processing */ int row, col; int img_index = 0; for (row = 0; row < image_array[window_index].height; row ++) { for (col = 0; col < image_array[window_index].width; col++) { /* copy picture array pics->pix_array to image_array.pixs */ image_array[window_index].pixs[img_index] = pic->pix_array[row][col]; /* write Pixel to socket for processing in openGL window */ write (image_array[window_index].socket_id[1], &(image_array[window_index].pixs[img_index].R), 1); write (image_array[window_index].socket_id[1], &(image_array[window_index].pixs[img_index].G), 1); write (image_array[window_index].socket_id[1], &(image_array[window_index].pixs[img_index].B), 1); img_index++; } } // printf ("image data sent to openGL process\n"); /* image number updated */ image_seq_number++; image_array[window_index].image_num = image_seq_number; /* wait until openGL process acknowledges receive of image data */ ch = '?'; int num_received = recv (image_array[window_index].socket_id[1], &ch, 1, MSG_WAITALL); printf (" main process: update: num__received: %d, code: %c\n", num_received, ch); pthread_mutex_unlock (&image_array[window_index].lock); /* check if update process timed out before updating new window */ if (num_received != 1) { /* if process quit, try again */ /* allow 2 attempts to create new window, in case existing processing is exiting as this one starts */ rDisplayPictureNewWindow (pic, duration, image_array[window_index].window_title, window_index, 2); } else { /* mark status of display window as having been made visible, but new image may or may not be processed (subject to functioning of update_display and glutPostRedisplay) */ image_array[window_index].process_status = 1; /* handle duration parameter for this image */ process_image_duration (duration, window_index); } printf (" main process: rDisplayPictureUpdate completed\n"); } // rDisplayPictureUpdate /** * add thread sequence number, id, etc. to thread_timer array; */ void recordTimerThread (int thread_seq_num, pthread_t thread) { // printf (" main process: registering thread with sequence number: %d\n", // thread_seq_num); int i; for (i = 0; i < current_size_thread_array; i++) { /* find unused array element, if present */ if (thread_info[i].thread_seq_number == 0) { /* insert thread record into array */ thread_info[i].thread_seq_number = thread_seq_num; thread_info[i].pthread = thread; return; } } /* no space found in previously-used part of thread array */ if (current_size_thread_array < max_size_thread_array) { /* insert thread record into array */ thread_info[current_size_thread_array].thread_seq_number = thread_seq_num; thread_info[current_size_thread_array].pthread = thread; current_size_thread_array++; return; } /* must extend thread array to accommodate more thread elements */ timing_thread_info * temp_thread_info = malloc (2 * max_size_thread_array * sizeof (image_timing_data)); for (i = 0; i < current_size_thread_array; i++) { temp_thread_info[i].thread_seq_number = thread_info[i].thread_seq_number; temp_thread_info[i].pthread = thread_info[i].pthread; } free (thread_info); max_size_thread_array *= 2; /* insert new thread info into array */ thread_info = temp_thread_info; thread_info[current_size_thread_array].thread_seq_number = thread_seq_num; thread_info[current_size_thread_array].pthread = thread; current_size_thread_array++; } /** * remove thread sequence number, id, etc. to thread_timer array; */ void deleteTimerThread (int thread_seq_num) { // printf (" main process: removing thread with sequence number: %d\n", // thread_seq_num); int i; for (i = 0; i < current_size_thread_array; i++) { /* if sequence number found */ if (thread_info[i].thread_seq_number == thread_seq_num) { /* delete record of thread */ thread_info[i].thread_seq_number = 0; return; } } } /* **** Implementations: MyroC functions ********************************* */ /** * @brief Display a picture * @param pic pointer to an RGB picture struct from Scribbler 2 camera * @param duration if duration > 0, operation is blocking \n * if duration <= 0, operation is non-blocking \n * for duration != 0, picture displayed for abs(duration) \n * seconds or until picture closed manually or * until the program terminates \n * if duration == 0, picture displayed until closed manually * @param windowTitle The title of the window that appears. * white spaces will be replaced with underscores. * @pre windowTitle is less than 100 characters. * @post image is displayed for the duration specified, \n * EXCEPT all display windows are closed * when the main program terminates. * @warning If images are displayed with a non-blocking option, and * if the user wants images to appear for a full duration, * use rWaitTimedImageDisplay or rFinishProcessing to block * processing until all image timers are finished. * Otherwise, program termination may close windows prematurely. */ void rDisplayPicture(Picture * pic, double duration, const char * windowTitle) { /* determine if window with given title exists */ for (window_index = 0; window_index < num_windows; window_index++) { if (strcmp (windowTitle, image_array[window_index].window_title) == 0) break; } /* 4 cases/subcases must be considered 1. window title was not encountered previously a. a full quota of named windows have already been created b. a new window can be created 2. window title was encountered previously a. a process currently is handling a window with that title b. the process previously handling the window no longer exists */ // printf ("rDisplayPicture: window_index: %d, num_windows: %d\n", // window_index, num_windows); if (window_index == num_windows) { /* case 1: check if window name previously used */ if (window_index >= max_windows) { /* case 1a: a full quota of named windows already created */ // printf ("--->case 1a: "); printf ("the number of requested image windows exceeds "); printf ("the maximum allowed (%d)\n", max_windows); printf (" image not displayed --- other processing continues\n"); return; } else { /*case 1b: a new named window can be created */ /* image title not previously recorded, so record image data in image_array */ // printf ("--->case 1b\n"); num_windows++; /* put window title in newly-allocated character array */ int title_len = strlen(windowTitle); image_array[window_index].window_title = (char *)malloc ((title_len + 1) * sizeof (char)); strncpy (image_array[window_index].window_title, windowTitle, title_len+1); /* record image size */ image_array[window_index].height = pic->height; image_array[window_index].width = pic-> width; image_array[window_index].image_num = 1; /* set up mutex lock, so communication for one image does not interfere with communication for a second image */ // printf ("set up mutex lock for window %d\n", window_index); if (pthread_mutex_init (&(image_array[window_index].lock), NULL) != 0) { printf ("\nmutex initialization failed\n"); exit (1); } /* create new openGL process to display picture */ /* allow 2 attempts to create new window, in case existing processing is exiting as this one starts */ // printf (" pid: %d; status: %d; errno: %d; err string: %s\n", // end_pid, status, errno, strerror (errno)); rDisplayPictureNewWindow(pic, duration, windowTitle, window_index, 2); return; } } else { /* 2. window title was encountered previously */ if (image_array[window_index].process_status == 1) { /* case 2a. a process currently handling window with given title */ /* double check that process ID is still active */ int status = -1; errno = 0; pid_t end_pid = waitpid (image_array[window_index].display_pid, &status, WNOHANG); if (!WIFEXITED(status) && (end_pid >= 0) && (errno != ECHILD)) { /* window exists and window process still running --- updating existing picture */ // printf ("--->case 2a---pid: %d; status: %d; errno: %d; err string: %s\n", // end_pid, status, errno, strerror (errno)); rDisplayPictureUpdate (pic, duration, window_index); return; } } /* case 2b. previous process handling window no longer exists */ /* create new openGL process to display picture */ /* allow 2 attempts to create new window, in case existing processing is exiting as this one starts */ // printf ("--->case 2b: %s\n", strerror (errno)); rDisplayPictureNewWindow(pic, duration, windowTitle, window_index, 2); return; } } /** @brief Turn on and off the printing of error messasge for image processing * @note Some operating systems (e.g., Mac OX S High Sierra) display * an error message whenevern a new process for image display starts * @post This function toggles this printing on and off * Initially, printing is off */ void rDisplayImageProcessingErrors () { displayImageErrors = !displayImageErrors; } /** * @brief Wait until all timed, non-blocking image window timers are complete * @post wait until all timed [non-blocking] images have closed \n * robot motion is unaffected by this function * * @warning images opened with duration 0.0 do not close until * images are updated or until the program terminates \n * all other image windows are closed by this function */ void rWaitTimedImageDisplay () { int window_index; for (window_index = 0; window_index < num_windows; window_index++) { // check if timed window still waitng to time out if (image_array[window_index].process_status == 1) { // waint for window process to conclude window processing waitpid (image_array[window_index].display_pid, NULL, 0); } else { // window either closed or open and not timed // so nothing to do } } }