Pascal + AxRuntime

[deadwood]

@magorium

Opening a new thread here.

I guess we will have to do some experiments

My proposal would be the following: create a simplest Pascal program that does one visual thing - for example opens an Intuition window. It should have the smallest amount of code possible.

Once you have it, split the Pascal "main" into two functions:
1) "secondary" main where you have the test code
2) "real" main where you open libaxrt-4.0.so and call __set_runtime_env and then __kick_start passing "secondary" main as first parameter (same as in __runtimestartup)

Let's see what happens.

One caveat that I already though about:
Each Ax program operates on two ABIs - one is Linux ABI for calls that directly use Linux shared objects, the other is ABIv11 for Amiga API calls. In case of "C" this is handled by either macros in includes or by linklibs, so the same compiler can be used in both cases. I don't know how this will work for Pascal. If you have some sort of packages that intermediate between Pascal program Amiga API you might need to use these packages when "linking" a Linux Ax Pascal binary.

[magorium]

I guess we will have to do some experiments

Indeed, at least from my side. I do think this requires the top-most of my knowledge with regards to FPC, but let's see how far i am able to get along :-)

My proposal would be the following: create a simplest Pascal program that does one visual thing - for example opens an Intuition window. It should have the smallest amount of code possible.

Reading your message completely before writing this line of text, that already keeps me busy for a little while as ABIv11 has become available only recently in FPC trunk (thanks to charlie on request of ALB42). Therefor i need to update my FPC installation first (working on that right now) 

Secondly, i then need to have a look at the state of the Amiga API units to see if they match current/your AROS development tree (library offsets etc). I have no experience with developing for 64 bit AROS whatsoever, let alone that i am aware of the state of the individual FPC unit files (although i am sure ALB42 did his best).

The intuition example would then not be too complicated to realize and, when AROS 64 hosted is able to run on my setup i can even check if that works correctly beforehand.

I will see what happens when applying your suggestion(s) and report back.

You are indeed correct with regards to your caveat. afaik it is not possible to compile a 'single' program using different ABI's at the same compilation run (i've already  dropped the question to FPC devs on irc, as i myself am not aware of such possibility. I might need to turn to other resources as well).
afaik, there are some inner (undocumented) features inside FPC that allow for switching syscalls for example, but i do not know to what extend and/or if it is a global or local (per unit) feature or perhaps even depend on the target platform.

fwiw: It would be too early in this process to get into details with regards on how the Free Pascal Compiler works with object files, but i presume we have to look into that pretty quickly.

[deadwood]

Secondly, i then need to have a look at the state of the Amiga API units to see if they match current/your AROS development tree (library offsets etc). I have no experience with developing for 64 bit AROS whatsoever, let alone that i am aware of the state of the individual FPC unit files (although i am sure ALB42 did his best).

The intuition example would then not be too complicated to realize and, when AROS 64 hosted is able to run on my setup i can even check if that works correctly beforehand.

You can make your local AROS 64 hosted build using these instructions https://github.com/deadw00d/AROS/blob/master/INSTALL.md and the same "delete" workaround.

[magorium]

The same trick indeed worked for the hosted version of AROS.


I seem to need to delve into the FPC sources because right now, when i start a simple helloworld (only writing the text helloworld) then the terminal output from AROS presents me with:

Code: [Select]


./boot/linux/AROSBootstrap
[Bootstrap] RAM memory block allocated: 0x41e63000 - 0x45e63000 (67108864 bytes)
[Bootstrap] Entering kernel at 0x4099c000...
[KRN] Protecting host process stack (0x00007ffd35505a80 - 0x00007ffd3550fa80)
[KRN] Trap signal 11, SysBase 0000000042e65830, KernelBase 0000000042e66a58
    RSP=0000000043485f80  RBP=0000000043485fc0  RIP=00000000409b32af
    RAX=0000000000000000  RBX=0000000000000000  RCX=0000000042fecd30  RDX=0000000042e65830
    RDI=0000000000000000  RSI=0000000042e65830  RFLAGS=0000000000010202
    R8 =00000000438503e0  R9 =0000000040ad0efb  R10=0000000043486110  R11=0000000042fc81c0
    R12=0000000000000000  R13=0000000000000000  R14=0000000000000000  R15=0000000000000000
[KRN] Trap signal 11, SysBase 0000000042e65830, KernelBase 0000000042e66a58
    RSP=0000000043484b98  RBP=0000000000000000  RIP=00007f8af29ecde7
    RAX=0000000000000000  RBX=00005558edcc4d60  RCX=0000000000000000  RDX=0000000000000000
    RDI=00005558edcc4d60  RSI=0000000043484bd8  RFLAGS=0000000000010246
    R8 =00005558edcc4d78  R9 =0000000000000000  R10=000000000000007d  R11=0000000000000246
    R12=00005558edcc4d78  R13=0000000000000000  R14=0000000000000001  R15=00005558edcc4d60


################################################################################
#                               Software Failure!                              #
#                    Task : 0x0000000043399600 - helloworld                    #
#             Error: 0x80000002 - Hardware bus fault/address error             #
################################################################################
PC   : 0x00000000409B32AF
Module boot/linux/kernel Segment 1 .text (0x000000004099C000) Offset 0x00000000000172AF
Function Exec_49_FindTask (0x00000000409B329C) Offset 0x0000000000000013
CPU context:
RAX=0000000000000000  RBX=0000000000000000  RCX=0000000042FECD30  RDX=0000000042E65830
RSI=0000000042E65830  RDI=0000000000000000  RSP=0000000043485F80  RBP=0000000043485FC0
R8 =00000000438503E0  R9 =0000000040AD0EFB  R10=0000000043486110  R11=0000000042FC81C0
R12=0000000000000000  R13=0000000000000000  R14=0000000000000000  R15=0000000000000000
RIP=00000000409B32AF  RSP=0000000043485F80  RFLAGS=0000000000010202
Stack trace:
0x0000000042FC81E5 System:Work/fpc/helloworld Function SI_PRC_$$__FPC_PROC_START$POINTER$INT64$POINTER$$LONGINT + 0x0000000000000025
0x00000000409A955C boot/linux/kernel Function Exec_134_NewStackSwap + 0x0000000000000144
0x00000000433647A0 Address not found
0x0000000040ADF07F Libs/dos.library Segment 1 .text + 0x000000000001C0DF
0x0000000040ADF545 Libs/dos.library Function Dos_84_RunCommand + 0x000000000000033B
0x00000000430A53FC System:L/UserShell-Seg Segment 1 .text + 0x0000000000001113
0x00000000430A6A73 System:L/UserShell-Seg Segment 1 .text + 0x000000000000278A
0x00000000430A5EC9 System:L/UserShell-Seg Function checkLine + 0x00000000000000B5
0x00000000430A5AE3 System:L/UserShell-Seg Function interact + 0x0000000000000164
0x00000000430A7262 System:L/UserShell-Seg Function _shell_ShellStart + 0x000000000000022E
################################################################################
[Bootstrap] Entering kernel at 0x4099c000...
[KRN] Protecting host process stack (0x00007ffd35505a80 - 0x00007ffd3550fa80)


afaik SI_PRC_xxxx belong to the FPC startup-code. Perhaps i missed a specific define for compilation for the new ABI or something similar. I'll let you know as soon as i know more, unless you have something up your sleeves

edit: add helloworld files
edit2: indeed si_prc_xxx is part of the startup, see also here (https://github.com/fpc/FPCSource/tree/main/rtl/aros) for the original (asm/pascal) source files for each supported platform.
edit3: as a reminder for myself: https://github.com/fpc/FPCSource/commit/568d4de77ec6088683375ea8ad376972e03d3c95

[magorium]

Please ignore previous message. Apologies in case it caused any inconvenience.

It was a stupidity form my side. Kindly hinted by ALB42. I used the wrong (too old) sources in order to compile the Free Pascal Compiler, so the generated executables were doomed to fail beforehand 

I now got a helloworld and intuitionwindow example running on Aros hosted as intended. I shall try your suggestions tomorrow and report back.

[deadwood]

Glad to hear issues are resolved. Looking forward to see what happens next ^^

[magorium]

ok, so i got the following Pascal source:

Code: [Select]

program test2;


{$MODE OBJFPC}{$H+}


uses
  ctypes, dynlibs;


const
  n_axrt = './libaxrt-4.0.so';


  RT_STARTUP  = 3;
  RT_ABI      = 4;
  RT_VERSION  = 41;
  RT_REVISION = 1;
  RT_VER      = ( (RT_STARTUP shl 28) or (RT_ABI shl 24) or (RT_VERSION shl 8) or (RT_REVISION shl 0) );


var
  h_axrt : TLibHandle = 0;


  __set_runtime_env : procedure(x: cint); cdecl;
  __kick_start      : procedure(p: pointer; x: cint); cdecl;


var
  memlocation: pointer;
  funname    : string;




function __startup_entry(argstr: pchar; argsize: longint; SysBase: pointer): cint; cdecl;
begin
  // are we still in linux context here ?
  result := 0;
end;




// main pascal startup entry:
// keep in mind that before entering here, FPC os related startup-code
// has already mapped some basic functionality to the OS, and fpc
// initialization code was performed in order to setup the compiler
// e.g. memory manager etc.
// Also it is possible that fpc already had Pascal related code running
// prior to entering this main startup entry e.g. each unit from the
// uses clause can have initialization/finalization code that is
// processed respectively before and after the main program entry as
// shown here and correspond to the order in which they are declared
// in the uses clause (e.g. last used unit that has initialization code
// is processed first and first used unit that has initialization code
// is processed last. The same for finalization sections but in reversed
// order).


begin
  writeln('begin');


  // load dynlib libaxrt
  h_axrt := LoadLibrary(n_axrt);


  // if libaxrt loaded successfully
  if h_axrt <> NilHandle then
  begin
    writeln('loaded ax runtime library');


    // #######################


    writeln('retrieving libaxrt function pointers');


    funname := '__set_runtime_env';
    memlocation := GetProcedureAddress(h_axrt, funname);
    if assigned(memlocation) then  pointer(__set_runtime_env) := memlocation else writeln('unable to locate function ' + funname);


    funname := '__kick_start';
    memlocation := GetProcedureAddress(h_axrt, funname);
    if assigned(memlocation) then  pointer(__kick_start) := memlocation else writeln('unable to locate function ' + funname);


    // assume everything went ok.


    writeln('set runtime environment');
    __set_runtime_env(RT_VER);


    writeln('kick start startup entry');
    __kick_start(@__startup_entry, RT_VER);


    // #######################


    if unloadlibrary(h_axrt)
      then writeln('unloaded ax runtime library')
    else writeln('failed unloading ax runtime library');
  end
  else writeln('failed to load ax runtime library');


  writeln('end');
end.


When i run this code in a fresh copy of axrt, executable located in AROS/Work/fpc then axrt outputs the following:

Code: [Select]

begin
loaded ax runtime library
retrieving libaxrt function pointers
set runtime environment
<<INFO>>: AxRT 41.1
<<INFO>>: Using absolute paths.
<<INFO>>: AXRT_ROOT environment variable set
<<INFO>>: RUNTIME_ROOT: /media/ramdisk/myrepo/alt-runtimelinux-x86_64-d/bin/runtimelinux-x86_64/AROS/
<<INFO>>: AXRTSYS     : ROOT:media/ramdisk/myrepo/alt-runtimelinux-x86_64-d/bin/runtimelinux-x86_64/AROS/
<<INFO>>: USERSYS     : ROOT:home/magorium/.axrt/
kick start startup entry
libaxrt-4.0.so: cannot open shared object file: No such file or directory
<<INFO>>: CURRENT_DIR : ROOT:media/ramdisk/myrepo/alt-runtimelinux-x86_64-d/bin/runtimelinux-x86_64/AROS/Work/fpc/
<<INFO>>: PROGRAM DIR : ROOT:media/ramdisk/myrepo/alt-runtimelinux-x86_64-d/bin/runtimelinux-x86_64/AROS/Work/fpc/
<<INFO>>: PROGRAM NAME: test2


relevant files attached (had to split it up due to board restrictions).

[deadwood]

Ok, two comments:

1) Where does "libaxrt-4.0.so: cannot open shared object file: No such file or directory" come from?
2) Your __start_entry has no code in it, try opening Intuition window in it

[magorium]

1) Where does "libaxrt-4.0.so: cannot open shared object file: No such file or directory" come from?

I was kind of hoping that you would have an idea about that 

For me it looks like libaxrt is trying to load something and is unable to locate it. I did a search on the axrt system install on my machine but i seem unable to locate that string there, so i am assuming that it is a error message that is returned from a linux function that is called from within libaxrt.so (either direct or indirect).


edit (added): also fpc does not seem to contain this string in any of it sources (i forgot to mention that in my initial post)

fwiw: if i open and close the shared object from linux and determine the function offsets  (so nothing else besides that) then the executable works as intended (outside axrt).

Could it perhaps originate from the fact that is use a relative path for loading libaxrt ?

2) Your __start_entry has no code in it, try opening Intuition window in it

It seems like  have already lost control.

I have no indication whatsoever that the kick start routine is actually invoked. Hence my remark in the source-code in which  context that function is called, aros or linux ? That way i hope i can try and output something useful somewhere in order to check if the routine is actually invoked or not.

[deadwood]

I have no indication whatsoever that the kick start routine is actually invoked. Hence my remark in the source-code in which  context that function is called, aros or linux ? That way i hope i can try and output something useful somewhere in order to check if the routine is actually invoked or not.

Well, its called in context of "AROS" and Linux. You can use Amiga API calls and Linux C library calls. Maybe you can also use Pascal calls?

[magorium]

Well, its called in context of "AROS" and Linux.

You can use Amiga API calls and Linux C library calls. Maybe you can also use Pascal calls?

I am able to confirm that yes i can use Pascal calls, be it in the context of Linux (fpc startup-code is Linux after all).

I am therefore also to confirm that the/my kick start routine is invoked \o/


With the tremendous help from ALB42 we are able to confirm that also the Amiga/AROS API side of things work as expected. I'll make up a more simplified and commented example and post it here later.

There is a concern however and that is that it does not seem that the startup_entry routine that is invoked by its call to kick_start ever returns to its origin, e.g. the code flow does not seem to return to the code directly after the call to kickstart.

That has some implication, namely that the fpc rtl does not shut down properly. You can check that with my example not outputting the writeln's and therefore also does not unload the dynamically loaded libaxrt and execute the rest of the (hidden RTL shutdown) code.

Is there any chance we are able can fix/adjust that or do you need more information from me/us for that to be able to answer first ?

TIA

[deadwood]

I am therefore also to confirm that the/my kick start routine is invoked \o/


With the tremendous help from ALB42 we are able to confirm that also the Amiga/AROS API side of things work as expected. I'll make up a more simplified and commented example and post it here later.

Does this mean you can write a Pascal program that will open an Intuition window under Linux now?

There is a concern however and that is that it does not seem that the startup_entry routine that is invoked by its call to kick_start ever returns to its origin, e.g. the code flow does not seem to return to the code directly after the call to kickstart.

That has some implication, namely that the fpc rtl does not shut down properly. You can check that with my example not outputting the writeln's and therefore also does not unload the dynamically loaded libaxrt and execute the rest of the (hidden RTL shutdown) code.

Is there any chance we are able can fix/adjust that or do you need more information from me/us for that to be able to answer first ?

TIA

That's actually a good point. The non-return was by design in the "old" startup code. With the code that I'm working right now, it might not be needed - that is the flow might be able to return "normally". I need to rethink that.

[magorium]

Does this mean you can write a Pascal program that will open an Intuition window under Linux now?

simple question, complicated answer 

ALB seem to have it running but i can't confirm as it is crashing for me (inside axrt) after the second system call. It looks like my sysbase is trashed or something similar, but i am not 100% sure.

There is a difference though. ALB42 is running 40.2 while I am running 41.1 (trunk). So what i will do is compile 40.2 from source, but without the utility directory and see if i am able to get the same results as ALB42 does.

That is, unless you deem it more useful to continue wit trunk and me offering my crash-logs ?

btw: on that subject, is smp enabled on axrt ? (because that is where my first crash happened inside lddemon).

That's actually a good point. The non-return was by design in the "old" startup code. With the code that I'm working right now, it might not be needed - that is the flow might be able to return "normally". I need to rethink that.

Ok. It is not a problem for right now but in the end it does cause some issues for FPC. No need to hurry with that though. (i rather see the avx issue fixed first  )

[deadwood]

There is a difference though. ALB42 is running 40.2 while I am running 41.1 (trunk). So what i will do is compile 40.2 from source, but without the utility directory and see if i am able to get the same results as ALB42 does.

That is, unless you deem it more useful to continue wit trunk and me offering my crash-logs ?

Yes, for me it is more useful to solve 41.1 problems. 40.2 was really experimental and it even has a different ABI. 41.1 ABI is the same as AROS x86-64, so it passes library base in R12. 40.2 was passing library base as last argument. This might be the actual trashing problem - what I mentioned earlier that you have to use units that are compiled for ABIv11 for AROS-side calls, not standard Linux ABI calls (standard SysV ABI).

btw: on that subject, is smp enabled on axrt ? (because that is where my first crash happened inside lddemon).

Yes it is, but it enabled in both 40.2 and 41.1.

[magorium]

Yes, for me it is more useful to solve 41.1 problems. 40.2 was really experimental

Ah ok. Just for the record 40.2 worked correctly for me as well. So the answer to the question you asked earlier, the answer is yes it works (but only for 40.2)

..and it even has a different ABI. 41.1 ABI is the same as AROS x86-64, so it passes library base in R12. 40.2 was passing library base as last argument. This might be the actual trashing problem

Aha. Yes that explains my crashes. I have not accounted for the new ABI in my current setup.

- what I mentioned earlier that you have to use units that are compiled for ABIv11 for AROS-side calls, not standard Linux ABI calls (standard SysV ABI).

That means back to the drawing board for me (although it was already discussed by me with ALB and he offered some suggestions, so i can at least make a start with that)


The problem there for me is that the FPC compiler currently either compiles for ABI no.1 or ABI no.2 and create an executable accordingly. The AROS x86_64 target has currently no provision to support both different syscall ABI's. Hence my earlier asking about the context when certain routines are being invoked.


It does mean i have to enable the new syscall ABI for the current AROS target for FPC and create a new compiler (the latter sounds more difficult than it actually is). For me undiscovered territory though, so might become interesting 

Yes it is, but it enabled in both 40.2 and 41.1.

Ok, good to know that it is/was intended. Thank you.


I'll report back once i got something working again. This might take me a while though, i have honestly no idea what troubles i might run into