Remote Debugging

This tutorial aims at providing approaches to remote debugging.


Remote debugging through ssh was not detailed but approached in Enlightenment Debugging section. In this section, it is more convenient to use ssh for debugging enlightenment because X is not available.

The version of SSH that you will want to use on Linux is called OpenSSH. To really make ssh useful, you need a shell account on a remote machine.

The first thing we'll do is simply connect to a remote machine. This is accomplished by running 'ssh hostname' on your local machine. The hostname that you supply as an argument is the hostname of the remote machine that you want to connect to. By default ssh will assume that you want to authenticate as the same user you use on your local machine. To override this and use a different user, simply use remoteusername@remotehostname or remoteusername@remoteipadress as the argument.

On your host machine:

ssh $remote_user_name@$remote_ip_address
ssh $remote_user_name@$remote_hostname

The first time around it will ask you if you wish to add the remote host to a list of known_hosts, go ahead and say yes.

The authenticity of host ' (' can't be established.
ECDSA key fingerprint is f0:72:8c:9d:aa:54:e4:5a:39:50:b5:b9:fd:35:71:71.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (ECDSA) to the list of known hosts.

It is important to pay attention to this question however because this is one of SSH's major features. Host validation. To put it simply, ssh will check to make sure that you are connecting to the host that you think you are connecting to. That way if someone tries to trick you into logging into their machine instead so that they can sniff your SSH session, you will have some warning.

Finally, your user password should be asked and you can execute your command as you were on the distant machine: Debugging Enlightenment using GDB.

You can also use ssh-keygen to generate RSA and DSA keys. Public key authentication for SSH sessions are far superior to any password authentication and provide much higher security. ssh-keygen is the basic way for generating keys for such kind of authentication.


gdbserver is a control program for Unix-like systems, which allows you to connect your program with a remote gdb via target remote but without linking in the usual debugging stub.

gdbserver is not a complete replacement for the debugging stubs, because it requires essentially the same operating-system facilities that gdb itself does. In fact, a system that can run gdbserver to connect to a remote gdb could also run gdb locally! gdbserver is sometimes useful nevertheless, because it is a much smaller program than gdb itself. It is also easier to port than all of gdb, so you may be able to get started more quickly on a new system by using gdbserver. Finally, if you develop code for real-time systems, you may find that the tradeoffs involved in real-time operation make it more convenient to do as much development work as possible on another system, for example by cross-compiling. You can use gdbserver to make a similar choice for debugging.

gdb and gdbserver communicate via either a serial line or a TCP connection, using the standard gdb remote serial protocol.

gdbserver does not have any built-in security. Do not run gdbserver connected to any public network; a gdb connection to gdbserver provides access to the target system with the same privileges as the user running gdbserver.

First, you have to install gdbserver on your server side.

Then, use a segfault program, you can use as in Segfault in callback function.

run gdbserver:

gdbserver localhost:2000 hello
$ Process hello created; pid = 2763
$ Listening on port 2000

Now gdbserver is waiting for a connexion on the port 2000.

<note> You could launch gdb in multiple process with gdbserver –multi localhost:2000. <\note>

On the client side, run gdb:

gdb hello
$ Type "apropos word" to search for commands related to "word"...
$ Reading symbols from test...(no debugging symbols found)...done.

Use target remotee to connect to the target system by indicating server ip and port.

(gdb)  target remote
$ Remote debugging using
$ Reading symbols from /lib64/ symbols from
$ /usr/lib/debug//lib/x86_64-linux-gnu/
$ done.
$ Loaded symbols for /lib64/
$ 0x00007ffff7ddb2d0 in _start () from /lib64/

At that point, you can add breakpoints or others, then if we continue our program, the output of the program will be printed in the target machine.

(gdb) continue

Now, the application is launched on server: if as i do you use the Segfault in callback function example: click on the seg button will provoke a segfault.

See after clicking, the traces on the client side:

$ Program received signal SIGSEGV, Segmentation fault.
$ 0x0000000000400b71 in ?? ()

You can now use gdb as it is explained in Segfault in callback function, for example print the backtrace:

(gdb) bt
$ 0  0x0000000000400b71 in ?? ()
$ 1  0x000000000073a9a0 in ?? ()
$ 2  0x00007ffff74e4f6c in _eo_evas_smart_cb (data=<optimized out>, eo_obj=<optimized out>, desc=<optimized out>, event_info=<optimized out>) at lib/evas/canvas/evas_object_smart.c:65
$ 3  0x00007ffff561a8b7 in _eo_base_event_callback_call (obj_id=0x80000007e0000040, pd=0x73a3f0, desc=0x85d4c0, event_info=0x0) at lib/eo/eo_base_class.c:712
$ 4  0x00007ffff5619648 in eo_event_callback_call (desc=desc@entry=0x85d4c0, event_info=event_info@entry=0x0) at lib/eo/eo_base.eo.c:94
$ 5  0x00007ffff74e6f6e in evas_object_smart_callback_call (eo_obj=eo_obj@entry=0x80000007e0000040, event=event@entry=0x7ffff7af501f <SIG_CLICKED> "clicked", event_info=event_info@entry=0x0) at lib/evas/canvas/evas_object_smart.c:784
$ 6  0x00007ffff7995baf in _activate (obj=0x80000007e0000040) at elm_button.c:69
$ 7  0x00007ffff7995be5 in _on_clicked_signal (data=<optimized out>, obj=<optimized out>, emission=<optimized out>, source=<optimized out>) at elm_button.c:191
$ 8  0x00007ffff58b0507 in edje_match_callback_exec_check_finals (prop=<optimized out>, ed=0xaaaaaaaaaaaaaaab, source=0x7245b0 "elm", sig=0x73af9c "elm,action,click", source_states=<optimized out>, signal_states=<optimized out>, matches=0x73b0c0, ssp=0x7594e0) at lib/edje/edje_match.c:556
$ 9  edje_match_callback_exec (ssp=ssp@entry=0x7594e0, matches=0x73b0c0, sig=sig@entry=0x73af9c "elm,action,click", source=source@entry=0x7245b0 "elm", ed=ed@entry=0x73a9a0, prop=prop@entry=0 '\000') at lib/edje/edje_match.c:711
$ 10 0x00007ffff58b6a3d in _edje_emit_cb (prop=0 '\000', data=0x0, src=0x7245b0 "elm", sig=0x73af9c "elm,action,click", ed=0x73a9a0) at lib/edje/edje_program.c:1423
$ 11 _edje_emit_handle (ed=0x73a9a0, sig=0x73af9c "elm,action,click", src=0x7245b0 "elm", sdata=0x0, prop=0 '\000') at lib/edje/edje_program.c:1376
$ 12 0x00007ffff58b13a1 in _edje_message_process (em=0x667830) at lib/edje/edje_message_queue.c:684
$ 13 0x00007ffff58b18b3 in _edje_message_queue_process () at lib/edje/edje_message_queue.c:787
$ 14 0x00007ffff58b1a14 in _edje_job (data=<optimized out>) at lib/edje/edje_message_queue.c:154
$ 15 0x00007ffff7235a32 in _ecore_job_event_handler (data=<optimized out>, type=<optimized out>, ev=<optimized out>) at lib/ecore/ecore_job.c:121
$ 16 0x00007ffff7230463 in _ecore_call_handler_cb (event=<optimized out>, type=<optimized out>, data=<optimized out>, func=<optimized out>) at lib/ecore/ecore_private.h:386
$ 17 _ecore_event_call () at lib/ecore/ecore_events.c:565
$ 18 0x00007ffff7237e05 in _ecore_main_loop_iterate_internal (once_only=once_only@entry=0) at lib/ecore/ecore_main.c:1927
$ 19 0x00007ffff7237f14 in ecore_main_loop_begin () at lib/ecore/ecore_main.c:983
$ 20 0x00007ffff7a47e59 in elm_run () at elm_main.c:1097


An UI inspection tool for the EFL. This tool lets you inspect UI elements and get a lot of their properties, e.g position, size and weight. Get more information there: Clouseau

Clouseau consists of a client (clouseau_client) and launcher (clouseau_server). This design means we can: Run the application we are debugging on one device and the clouseau_client itself on another. It is very similar to gdbserver.

Usage of the split client/server approach (different machines/environments/setups):

Server side

Get the ip of your server machine, for example do:

ip a

Then start, the server side with the clouseau_start script launching the application:

clouseau_start elementary_test
clouseau_start ./your_application
Client side

Simply do:

clouseau_client $ip_server

or use the client interface as below:

And now, clouseau is usable as in Clouseau.