I'm working on automatic C++ code testing using Python. So I have a Python script that compiles and executes C++ code. When the C++ code crashs, libc output is visible from my Python script output, even if I redirected cout
and cerr
of the C++ program being executed to /dev/null
.
Here is a sample isolating the problem:
#! /usr/bin/env python
# -*- coding: utf8 *-*
import sys
import os.path
import subprocess
import shutil
import tempfile
import string
import argparse
bug_folder = tempfile.mkdtemp("bug")
# Create main.cpp
with open( os.path.join(bug_folder,'main.cpp'), "w") as mainfile:
mainfile.write( "int main()\n" )
mainfile.write( "{\n" )
mainfile.write( " int* toto = new int(4);\n" )
mainfile.write( " delete toto;\n" )
mainfile.write( " delete toto;\n" )
mainfile.write( "}" )
# Create Makefile
project = "bug"
with open( os.path.join(bug_folder,'Makefile'), "w") as makefile:
makefile.write( "PROG = " + project + "\n" )
makefile.write( "OBJS = main.o\n" )
makefile.write( "CC = g++\n" )
makefile.write( "CFLAGS = -Wall -g -pedantic\n" )
makefile.write( "\n" )
makefile.write( "$(PROG): $(OBJS)\n" )
makefile.write( "\t$(CC) $(CFLAGS) -o $@ $(OBJS)\n" )
makefile.close()
# Compile
delete_folder = True
old_path = os.getcwd()
os.chdir(bug_folder)
if subprocess.call(["make"]) == 0 and os.path.isfile( project ):
# Project could be compiled!!
print "Running program..."
with open( os.devnull, "w") as outfile:
with open( os.devnull, "w") as errfile:
subprocess.call( [os.path.join(bug_folder,project)], stdout=outfile, stderr=errfile )
print "Done"
else:
print "Failed to compile (" + bug_folder + ")"
delete_folder = False
os.chdir(old_path)
if delete_folder:
shutil.rmtree( bug_folder )
Program output is:
g++ -c -o main.o main.cpp
g++ -Wall -g -pedantic -o bug main.o
Running program...
*** glibc detected *** /tmp/tmpxX20mLbug/bug: double free or corruption (fasttop): 0x00000000006cc010 ***
======= Backtrace: =========
/lib64/libc.so.6[0x328c275e66]
/tmp/tmpxX20mLbug/bug[0x40063b]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x328c21ed5d]
/tmp/tmpxX20mLbug/bug[0x400549]
======= Memory map: ========
00400000-00401000 r-xp 00000000 00:12 93358464 /tmp/tmpxX20mLbug/bug
00600000-00601000 rw-p 00000000 00:12 93358464 /tmp/tmpxX20mLbug/bug
006cc000-006ed000 rw-p 00000000 00:00 0 [heap]
328be00000-328be20000 r-xp 00000000 fd:00 136549 /lib64/ld-2.12.so
328c01f000-328c020000 r--p 0001f000 fd:00 136549 /lib64/ld-2.12.so
328c020000-328c021000 rw-p 00020000 fd:00 136549 /lib64/ld-2.12.so
328c021000-328c022000 rw-p 00000000 00:00 0
328c200000-328c38a000 r-xp 00000000 fd:00 136550 /lib64/libc-2.12.so
328c38a000-328c58a000 ---p 0018a000 fd:00 136550 /lib64/libc-2.12.so
328c58a000-328c58e000 r--p 0018a000 fd:00 136550 /lib64/libc-2.12.so
328c58e000-328c58f000 rw-p 0018e000 fd:00 136550 /lib64/libc-2.12.so
328c58f000-328c594000 rw-p 00000000 00:00 0
328c600000-328c683000 r-xp 00000000 fd:00 136551 /lib64/libm-2.12.so
328c683000-328c882000 ---p 00083000 fd:00 136551 /lib64/libm-2.12.so
328c882000-328c883000 r--p 00082000 fd:00 136551 /lib64/libm-2.12.so
328c883000-328c884000 rw-p 00083000 fd:00 136551 /lib64/libm-2.12.so
390fa00000-390fa16000 r-xp 00000000 fd:00 131093 /lib64/libgcc_s-4.4.7-20120601.so.1
390fa16000-390fc15000 ---p 00016000 fd:00 131093 /lib64/libgcc_s-4.4.7-20120601.so.1
390fc15000-390fc16000 rw-p 00015000 fd:00 131093 /lib64/libgcc_s-4.4.7-20120601.so.1
390fe00000-390fee8000 r-xp 00000000 fd:00 658079 /usr/lib64/libstdc++.so.6.0.13
390fee8000-39100e8000 ---p 000e8000 fd:00 658079 /usr/lib64/libstdc++.so.6.0.13
39100e8000-39100ef000 r--p 000e8000 fd:00 658079 /usr/lib64/libstdc++.so.6.0.13
39100ef000-39100f1000 rw-p 000ef000 fd:00 658079 /usr/lib64/libstdc++.so.6.0.13
39100f1000-3910106000 rw-p 00000000 00:00 0
7f302c4bb000-7f302c4c0000 rw-p 00000000 00:00 0
7f302c4e5000-7f302c4e7000 rw-p 00000000 00:00 0
7fffc6840000-7fffc6856000 rw-p 00000000 00:00 0 [stack]
7fffc6a00000-7fffc6a01000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Done
I would prefer:
g++ -c -o main.o main.cpp
g++ -Wall -g -pedantic -o bug main.o
Running program...
Done
Is it possible to make execution not show the whole glibc error report?
Note: I tried set LIBC_FATAL_STDERR_=1
in my shell before running the python script but it did not fix the problem at all.
I'm using Python 2.7.4 and g++ 4.4.7
Answer
The glibc error messages you're seeing are printed directly to stderr
, and unfortunately, redirecting stdout
and stderr
in the way you're doing may not always prevent these internal messages from appearing. One possible way to suppress these messages is by setting the LIBC_FATAL_STDERR_
environment variable to 0
before running the C++ program. However, as you mentioned, this didn't work for you.
You can try a few other approaches to capture or suppress the glibc errors:
1. Redirect stderr
in Python (again)
You may want to redirect both stderr
and stdout
to /dev/null
in Python explicitly when calling subprocess.call()
. For example:
import subprocess
import os
with open(os.devnull, 'w') as devnull:
subprocess.call([os.path.join(bug_folder, project)], stdout=devnull, stderr=devnull)
If that doesn't work, it might be because the glibc errors are being printed to the terminal directly, not through standard output or error.
2. Use os.environ
to try setting the environment variable
In addition to setting the LIBC_FATAL_STDERR_
variable in the shell, try setting it within the Python script before calling the subprocess:
import os
import subprocess
# Set environment variable to suppress glibc fatal stderr
os.environ["LIBC_FATAL_STDERR_"] = "0"
with open(os.devnull, 'w') as devnull:
subprocess.call([os.path.join(bug_folder, project)], stdout=devnull, stderr=devnull)
This method forces the environment variable setting within the Python process.
3. Use os.setsid()
to start a new session
Sometimes running the subprocess in a new session can prevent certain types of output from being printed to the terminal:
import os
import subprocess
# Start the program in a new session to avoid terminal output
subprocess.call(["setsid", os.path.join(bug_folder, project)], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
4. Catch and handle errors in the C++ program
If you're able to modify the C++ code, consider using signal()
to handle signals more gracefully, or try catching the error in C++ and suppressing it directly in your program, so that it's not passed through stderr
.
5. Suppress output at the libc level (less reliable)
If nothing works, you could also try disabling glibc's internal error messages by running the program with the following environment variable:
export MALLOC_PERTURB_="0"
This will suppress some types of memory corruption errors, though it may not help in all cases.
Let me know if one of these solutions works or if you encounter any issues!