Running and communicating with cli-programs from a wb progra
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    I'm currently learning to program intuition and everything to make an interface for chess engines. Chess engines are typically cli-programs. But I am not sure how to handle them from the GUI.

    What's the best way to do this on Amiga OS?

    The typical execution loop would be :

    Start:
    load engine
    Exec Loop:
    -> send commands to engine
    <- get result from engine and display them in the GUI
    Goto Exec Loop

    Eventually, one of the command would be "quit". All this has to be in separate processes (or threads ?) as the GUI has to be responsive at any time. For instance, if the user decides to quit from the menu or the close gadget, the GUI must be able to send a "quit" command to the engine even if it is currently computing.

    Also, there are situations when several engines could be loaded and communicating with, like an engine vs engine tournament, for instance. So I guess they'd need separate cli-handling ?

    I hope the description is enough. So, what's the best way to do that ? I see a couple of functions in the dos.library that I guess are possible candidates : Cli, CreateProc, Execute, SetConsoleTask, etc...

    I already ported my chess engine last year to MorphOS, but it was a pure cli program and so had nothing specifically amigaish in it. Now a GUI is something totally different. :)
    "A good bug is a dead bug" (Don Dailey)
  • »09.04.14 - 19:24
    Profile Visit Website
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    ausPPC
    Posts: 543 from 2007/8/6
    From: Pending...
    There are programing tutorials you might like to look at in the library section of Morphzone but I remember spending some time playing with this aminet archive and finding it useful too - http://aminet.net/package/dev/src/DisKoTutorial
    PPC assembly ain't so bad... ;)
  • »09.04.14 - 22:08
    Profile Visit Website
  • MorphOS Developer
    jacadcaps
    Posts: 3108 from 2003/3/5
    From: Canada
    Quote:

    JuLieN wrote:
    I already ported my chess engine last year to MorphOS, but it was a pure cli program and so had nothing specifically amigaish in it. Now a GUI is something totally different. :)


    I wouldn't start from going this low level, into exec or intuition - that is going to be a steep learning curve to say the least. Why not go with MUI and offload your engine work into a Slave.mui class? Making a MUI application is a lot easier than using intuition.
  • »10.04.14 - 07:14
    Profile Visit Website
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    So, in two words : nobody knows ? ;)

    How fast is MUI, compared with vanilla Intuition ? chess GUIs are very speed-critical: they must take as less CPU time as possible to leave all of it to the engines. Engines send kilobytes of informations back to the GUI per second : current move examined, how many nodes traveled through, time used, best line, etc... The GUI received those informations, parse them and use them accordingly, like for instance by displaying the current best variation to the user, etc... So it has to use the fastest functions the OS can provide.

    Maybe I'm wrong, but it is my understanding that MUI sits on top of Intuition, and so adds a layer to the OS and must then be a bit slower than vanilla Intuition.

    But my problem, really, isn't the GUI itself, it's communicating with a cli-program : sending it commands and receiving its output back, while keeping the communication canal opened throughout the whole game.
    "A good bug is a dead bug" (Don Dailey)
  • »10.04.14 - 11:17
    Profile Visit Website
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    To be less abstract, here's an example of session in a real shell :
    morphoschess.jpg
    Playing the role the GUI should play, I've typed the following commands : uci, ucinewgame, go infinite, stop, quit.
    Everything else is the engine's answer and has to be parsed by the GUI.
    "A good bug is a dead bug" (Don Dailey)
  • »10.04.14 - 11:37
    Profile Visit Website
  • MorphOS Developer
    jacadcaps
    Posts: 3108 from 2003/3/5
    From: Canada
    Quote:

    JuLieN wrote:
    So, in two words : nobody knows ? ;)



    Oh we do know, but what we don't know is the structure of your program, the way you want the engine to communicate with the GUI, etc., etc.

    MUI vs intuition speed difference is negligible in your case - you are going to waste most CPU time passing and parsing information in text format from your cli command. This is where you are going the wrong way - having the engine in a separate cli application is a mistake. The GUI and the engine should be integrated into one application.
  • »10.04.14 - 11:48
    Profile Visit Website
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    @jacadcaps
    You don't understand. :) Having the engine and the GUI in the same program would be very easy, of couse. But the idea is to add engines when there are new ones compiled. And there exist hundreds of engines. The user could compile new engines that weren't already compiled, and add them to his collection. This is how ALL chess interfaces proceed. For instance :
    - Windows : http://www.playwitharena.com
    - Linux : http://www.gnu.org/software/xboard/
    - OSX : http://www.hiarcs.com/chess-explorer.htm

    Etc...

    Well, there's none for Amiga OS. Only programs dedicated to one program (like AmiChess, running GNU Chess.) And, by the way, GNU Chess is very weak, compared with modern state of the art engines. Here's the current chess engines charts (at least for the top engines) :
    http://www.computerchess.org.uk/ccrl/4040/

    As you can see, the best version of GNU Chess (actually a Fruit derivative, as Fabien Letouzey gave his code to the GNU foundation, to replace the even weaker previous GNU Chess versions), is only at the 49th place, nearly 600 Elo points behind the best one, Houdini.

    To put things into perspective :

    - GNU Chess : 2777 Elo
    - Gary Kasparov : 2852 Elo at his peek
    - Deep Blue, IBM's monster that defeated Kasparov : around 2860 Elo
    - Magnus Carlsen, current world Champion : 2882 Elo
    - Stockfish (best free and open source program) : 3249 Elo
    - Houdini (best commercial program) : 3273 Elo

    I hope you understand the interest of having such an interface, now. :)

    Back to my problem : what's the best way to get a wb program open a communication canal with a cli-program ?
    "A good bug is a dead bug" (Don Dailey)
  • »10.04.14 - 12:13
    Profile Visit Website
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    Tcheko
    Posts: 534 from 2003/2/25
    From: France
    Have a look to SystemTagList in dos.aros.guide in the SDK.

    Simply capture the IN: and the OUT: of the command line to communicate between your MUI app and the command line engine. Specifically : SYS_Output and SYS_Input.

    And go for MUI. Do not mess with Intuition.
    Quelque soit le chemin que tu prendras dans la vie, sache que tu auras des ampoules aux pieds.
    -------
    I need to practice my Kung Fu.
  • »10.04.14 - 12:35
    Profile Visit Website
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    Thanks Tcheko! :) I'll take a look now.
    "A good bug is a dead bug" (Don Dailey)
  • »10.04.14 - 12:36
    Profile Visit Website
  • MorphOS Developer
    itix
    Posts: 1520 from 2003/2/24
    From: Finland
    Intuition is so level that extra development time due to its low level complexity exceeds minor overhead added by higher level MUI layer.

    It is better create GUI and only after that benchmark its possible bottlenecks and then optimize it. Code first, optimize second. Not vice versa.
    1 + 1 = 3 with very large values of 1
  • »10.04.14 - 13:55
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    eliot
    Posts: 565 from 2004/4/15
    To control the chess engine your just have to start them from your program in
    a new process and get the std_in and std_out of the newly started process.
    Than you can send commands (std_in) and parse the response (std_out) .
    The GUI framework has nothing to do with that.

    regards
    eliot

    [ Edited by eliot 11.04.2014 - 09:02 ]
    regards
    eliot
  • »10.04.14 - 16:05
    Profile
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    @Eliot:
    yes, the Intuition/MUI discussion is really irrelevant and I've been trying to focus this thread back to my real problem. :)

    What do you mean exactly ? How would you start the engine from the GUI then ? Execute() doesn't seem to be interactive, more like a "fire and forget" function. What would you use ? RunCommand() ? CreateNewProc maybe ? Sorry for the naive questions, I'm really a beginner hone it comes to Amiga system programming. :)

    I've taken a look at SystemTagList() and it looks quite complex to use. Also I've read on a forum (maybe incorrectly ?) that it'll open a wheel window, when what I want to do has to be invisible to the user. I read this here, from someone who obviously was trying to solve the same problem :
    http://aros-exec.org/modules/newbb/viewtopic.php?start=0&topic_id=7987&viewmode=flat&order=ASC&type=&mode=0
    "A good bug is a dead bug" (Don Dailey)
  • »10.04.14 - 16:21
    Profile Visit Website
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    connor
    Posts: 570 from 2007/7/29
    http://aminet.net/package/game/board/AmiChessMOS-2.1 comes with the source. You can try how fast it is using MUI. Then you do not have to write every bit of the program again. And maybe it can be optimized later.
  • »10.04.14 - 17:48
    Profile
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    @Connor:
    Thanks, I've already checked AmiChess' sources. Amichess doesn't run an external exe, in contrary to what I'm seeking to do. The Chess engine and the GUI are one and single exe. What I want to do is a GUI that can load external chess engines. (And sometimes two different engines at the same time, when running engine vs engine tournament, for instance).
    "A good bug is a dead bug" (Don Dailey)
  • »10.04.14 - 18:05
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    igracki
    Posts: 416 from 2003/2/24
    From: Berlin
    you could write the engines as plugins (libraries) instead of external programs, the communication would be easier as with external programs.
  • »11.04.14 - 00:07
    Profile Visit Website
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    eliot
    Posts: 565 from 2004/4/15
    Have a look at System() function.
    This should help: System()
    Currently my MorphOs machine is turned off but there should be a similar function on MorphOs.

    [ Edited by eliot 11.04.2014 - 09:09 ]
    regards
    eliot
  • »11.04.14 - 07:06
    Profile
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    @Eliot:

    Thanks a lot, we're making progress : it's part of stdlib and it launches the engine. :) I was searching for Amiga-specific functions when the standard one was available!

    The only problem now is that, if the GUI is run from the shell it displays the engine output in it, or if it is run from the wb, even worse, it opens it's own output window. I'll have to find a way to redirect input and output I guess.
    "A good bug is a dead bug" (Don Dailey)
  • »11.04.14 - 08:35
    Profile Visit Website
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    Ok, I'm still stuck with this problem... How do you "get the std_in and std_out of the newly started process" ? :(
    "A good bug is a dead bug" (Don Dailey)
  • »16.04.14 - 19:13
    Profile Visit Website
  • MorphOS Developer
    jacadcaps
    Posts: 3108 from 2003/3/5
    From: Canada
    Quote:

    JuLieN wrote:
    Ok, I'm still stuck with this problem... How do you "get the std_in and std_out of the newly started process" ? :(


    Well... as piping isn't very reliable, I can only suggest that you prepare your own DOS handler for input and output and handle read and write DOS packets in it. A handler gives you a universal way of handling I/O (both a console and every file-system are handlers).

    You could check http://aminet.net/package/comm/misc/AuxHandler for an example - not directly usable but should give you an idea.

    [ Edited by jacadcaps 17.04.2014 - 06:02 ]
  • »17.04.14 - 03:56
    Profile Visit Website
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    Thanks, I'll take a look. I've also asked H.G. Müller, the current maintainer of XBoard/Winboard, and about piping he told me:

    Quote:

    The trick is that you create the pipes first. Each pipe has an input and an output, described by low-level file descriptors (numbers). You need two pipes, one for GUI->engine, one for engine->GUI. After you created the pipes, you fork. The child process inherits the open file descriptors. The trick is now to to make sure the file-descriptors at the engine side become 0 and 1, for the engine stdin and stdout. For this the dup() call is used to 'duplicate' a file descriptor (actually giving you a new one referring to the same file). After you set thet up, you use execv() in the child process to start the engine executable (which then inherits the file descriptors).


    This is of course for Linux, but would that take care of the reliability problem you mentioned for MorphOS ? As you can guess I'd rather do it the "Amiga" way, for MorphOS, so I'll take a look at your hint first. :)

    I'd never had guessed it would be so tricky on MorphOS, when it's so easy on OSX/Linux... Well, no pain no merit I guess. ;)
    "A good bug is a dead bug" (Don Dailey)
  • »17.04.14 - 08:57
    Profile Visit Website
  • MorphOS Developer
    jacadcaps
    Posts: 3108 from 2003/3/5
    From: Canada
    ixemul pipes of course work to some degree, but that'd only solve your issue for the backend part - the frontend (GUI) app cannot use ixemul.
  • »17.04.14 - 12:06
    Profile Visit Website
  • Caterpillar
    Caterpillar
    JuLieN
    Posts: 35 from 2008/4/16
    From: France
    I didn't know that. That pretty much kills the pipes-way. It wasn't amigaish at all anyway, so I already discarded it. I really want to do this the Amiga way.

    Right now I'm considering the invisible console ("CONU_LIBRARY") idea. Do you think it could work ? The general idea would be :

    -> open invisible console
    -> create the engine's process
    -> attach the invisible console to this process
    "A good bug is a dead bug" (Don Dailey)
  • »17.04.14 - 12:34
    Profile Visit Website
  • MorphOS Developer
    jacadcaps
    Posts: 3108 from 2003/3/5
    From: Canada
    The "invisible console" is the handler I mentioned earlier. How would you grab the output from an invisible console? If you prepare a handler, then all the output goes to you via ACTION_WRITE.
  • »18.04.14 - 08:17
    Profile Visit Website