Segmentation fault [Solution found: use rock-convert]

[InigoMoreno]

Hello, I was trying to transform a log to msgpack and I got a segmentation fault. The file is included in this zip file.

I have been messing around to try to find the source of the error and I suspect the segmentation fault happens in this line.
Typelib::load( val, data.data(), data.size() );

[AlexanderFabisch]

Hi @InigoMoreno ,

I will take a look at it soon. Maybe @vbargsten has a better idea of what is happening here, since he recently reimplemented this part.

[AlexanderFabisch]

After a first quick look at the logfile, here seems to be the error:

$ docker run --interactive --rm --tty --volume "$PWD":/wd --workdir /wd -it af01/pocolog2msgpack -l waypoint_navigation.log --only /gps.pose_samples -v 5
[pocolog2msgpack] Verbosity level is 5.
[pocolog2msgpack] Only converting port '/gps.pose_samples'.
[pocolog2msgpack] 1 streams
[pocolog2msgpack] Stream #0 (/gps.pose_samples): 25392 samples
[pocolog2msgpack] Converting sample #0
[pocolog2msgpack] Converting sample of size 429
20  ee  29  0b  8a  51  05  00  04  00  00  00  00  00  00  00  62  6f  64  79  09  00  00  00  00  00  00  00  77  6f  72  6c  
64  5f  6f  73  67  00  48  5f  8a  65  17  65  40  00  00  f3  c4  bb  50  6e  40  00  48  e1  7a  14  ae  01  c0  a3  60  d6  
15  eb  43  f5  3e  60  08  07  d5  11  da  d2  3e  00  00  00  00  00  00  00  00  60  08  07  d5  11  da  d2  3e  18  d0  fb  
75  ef  b9  f5  3e  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  c1  93  16  
2e  ab  ...

Could you say what kind of data you expect in the first sample of the stream /gps.pose_samples?

[InigoMoreno]

/gps.pose_samples should contain values of gps data, with valid position and orientation. You can actually successfully run pocolog on it, and even replay it on a ruby script

pocolog waypoint_navigation.log -s /gps.pose_samples --to 0 gives us the first sample:

pocolog.rb[INFO]: loading file info from /media/heimdal/Dataset1/9June/Traverse/20170609-1909/waypoint_navigation.idx... 
/gps.pose_samples.time.microseconds /gps.pose_samples.sourceFrame /gps.pose_samples.targetFrame /gps.pose_samples.position.data[0] /gps.pose_samples.position.data[1] /gps.pose_samples.position.data[2] /gps.pose_samples.cov_position.data[0] /gps.pose_samples.cov_position.data[1] /gps.pose_samples.cov_position.data[2] /gps.pose_samples.cov_position.data[3] /gps.pose_samples.cov_position.data[4] /gps.pose_samples.cov_position.data[5] /gps.pose_samples.cov_position.data[6] /gps.pose_samples.cov_position.data[7] /gps.pose_samples.cov_position.data[8] /gps.pose_samples.orientation.im[0] /gps.pose_samples.orientation.im[1] /gps.pose_samples.orientation.im[2] /gps.pose_samples.orientation.re /gps.pose_samples.cov_orientation.data[0] /gps.pose_samples.cov_orientation.data[1] /gps.pose_samples.cov_orientation.data[2] /gps.pose_samples.cov_orientation.data[3] /gps.pose_samples.cov_orientation.data[4] /gps.pose_samples.cov_orientation.data[5] /gps.pose_samples.cov_orientation.data[6] /gps.pose_samples.cov_orientation.data[7] /gps.pose_samples.cov_orientation.data[8] /gps.pose_samples.velocity.data[0] /gps.pose_samples.velocity.data[1] /gps.pose_samples.velocity.data[2] /gps.pose_samples.cov_velocity.data[0] /gps.pose_samples.cov_velocity.data[1] /gps.pose_samples.cov_velocity.data[2] /gps.pose_samples.cov_velocity.data[3] /gps.pose_samples.cov_velocity.data[4] /gps.pose_samples.cov_velocity.data[5] /gps.pose_samples.cov_velocity.data[6] /gps.pose_samples.cov_velocity.data[7] /gps.pose_samples.cov_velocity.data[8] /gps.pose_samples.angular_velocity.data[0] /gps.pose_samples.angular_velocity.data[1] /gps.pose_samples.angular_velocity.data[2] /gps.pose_samples.cov_angular_velocity.data[0] /gps.pose_samples.cov_angular_velocity.data[1] /gps.pose_samples.cov_angular_velocity.data[2] /gps.pose_samples.cov_angular_velocity.data[3] /gps.pose_samples.cov_angular_velocity.data[4] /gps.pose_samples.cov_angular_velocity.data[5] /gps.pose_samples.cov_angular_velocity.data[6] /gps.pose_samples.cov_angular_velocity.data[7] /gps.pose_samples.cov_angular_velocity.data[8]
1497028218187296 98 111 100 121 119 111 114 108 100 95 111 115 103 168.73114508256549 242.52292106114328 -2.2100000000000364 2.0280176035920025e-05 4.4946276180364683e-06 0 4.4946276180364683e-06 2.0719823964079979e-05 0 0 0 0.00019600000000000002 0 0 -0.39394190959093656 -0.91913533925524071 0 0 0 0 0 0 0 0 1918.4399999999998 nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan

[InigoMoreno]

Update, I ran rock-convert waypoint_navigation.log and then pocolog2msgpack was successful on the updated logfile.

While I managed to resolve my issue, I think a more detailed error message should be added to help future users.

[AlexanderFabisch]

OK, thanks. Do you have any idea how we can detect the old log format? What does rock-convert actually do?

[InigoMoreno]

I found about rock-convert in this tutorial. Not sure where it is documented, but it seems to update the base-types in logs if they have been changed since the log was recorded.

As for detecting the old base-types I am not exactly sure, maybe we could check for the byte size of the type versus what is stored in the log, but I'm not sure how to do that.

[AlexanderFabisch]

OK, I think we can leave the issue open and I will add a hint to the readme. It has low priority for me at the moment so I'm not sure when I can take a closer look. I don't know how pocolog does it internally...

[vbargsten]

I checked yesterday and found that there is definitely a problem with the type initialization. At this point https://github.com/orocos-toolchain/typelib/blob/a247e11c8c7a3770fc5f81959a934178d1bf13d4/lang/csupport/containers.cc#L493, there is no valid string object in the memorylayout -- and thus clear or append won't work (it failed for a type of RigidBodyState with 2 strings being initialized).