Pascal + AxRuntime

[deadwood]

It does mean i have to enable the new syscall ABI for the current AROS target for FPC and create a new compiler (the latter sounds more difficult than it actually is). For me undiscovered territory though, so might become interesting 

Isn't there a way to pre-compile the "proxy" units (those who forward calls from Pascal to AROS) on AROS x86_64 and the use these pre-compiled units with normal Linux FPC? The only addition in AROS ABIv11 is that library base is placed into R12 register before making a call. Rest is normal SysV. In case of C this addition is handled for example in a linker library (= pre-compiled unit).

[deadwood]

I have a general question about FCP on x86_64 (Linux I guess). Which code model (small, medium, large) is the default for the FPC compiler?

[magorium]

I have a general question about FCP on x86_64 (Linux I guess). Which code model (small, medium, large) is the default for the FPC compiler?

uhm 


I had to look that up, and found this: https://eli.thegreenplace.net/2012/01/03/understanding-the-x64-code-models/


I have never heard of it in the context of FPC other than for 16 bit. It makes sense to me though 


But, i have honestly no idea. I've dropped the question on #fpc-dev but no response yet.


Currently working on modifying the compiler, but there seems more to it than meets the eye (as thought initially). I'll reply to your previous post later (not sure if that i am able to today though).

[magorium]

OK, i think i got the compiler to support AXRT ABIv11 syscalls for target Linux.

However, things did not go entirely as expected (probably my wrongdoing).

Given the following Free Pascal source-code (i cleaned up the empty space/comments a little before posting so the linenumber might be a bit off).

Code: (pascal) [Select]

program linuxaxrttest;

{$MODE OBJFPC}{$H+}

uses
  ctypes, dynlibs;

const
  n_axrt = './libaxrt-4.0.so';

  RT_STARTUP  = 3;
  RT_ABI      = 4;
  RT_VERSION  = 41;
  RT_REVISION = 1;
  RT_VER      = ( (RT_STARTUP shl 28) or (RT_ABI shl 24) or (RT_VERSION shl or (RT_REVISION shl 0) );

var
  h_axrt      : TLibHandle = 0;
  memlocation : pointer;
  funname     : string;

  __set_runtime_env : procedure(x: cint); cdecl;
  __kick_start      : procedure(p: pointer; x: cint); cdecl;

var
  AOS_ExecBase : Pointer;
  IntuitionBase: Pointer;
 
type
  // exec.library functionality

  ULONG    = LongWord;
  STRPTR   = pchar;
  PLibrary = Pointer;

  function  AvailMem(Requirements: ULONG): ULONG; syscall AOS_ExecBase 36;
  function  OpenLibrary(const LibName: STRPTR; Version: ULONG): PLibrary; syscall AOS_ExecBase 92;
  procedure CloseLibrary(Library_: PLibrary); syscall AOS_ExecBase 69;


function __startup_entry(argstr: pchar; argsize: longint; SysBase: pointer): cint; cdecl;
begin
  Result := 0;
  AOS_ExecBase := SysBase;
 
  writeln('hello axrt from fpc - still linux');

  writeln('before AvailMem');
  writeln('AvailMem(0) = ', Availmem(0));
  writeln('after AvailMem');

  writeln('are we still alive ?');

  writeln('before AvailMem');
  writeln('AvailMem(0) = ', Availmem(0));
  writeln('after AvailMem');

  writeln('still living ?');

  // open intuition library
  writeln('attempt to open library');
  IntuitionBase := OpenLibrary('intuition.library', 0);
  writeln('Open Library ', HexStr(IntuitionBase));
 
  // close intuition library
  writeln('attempt to close library');
  if assigned(IntuitionBase) then CloseLibrary(IntuitionBase);
end;

begin
  writeln('begin');

  // load dynlib libaxrt
  h_axrt := LoadLibrary(n_axrt);

  // if libaxrt loaded successfully
  if h_axrt <> NilHandle then
  begin
    writeln('loaded ax runtime library');

    // #######################

    writeln('retrieving libaxrt function pointers');
   
    funname := '__set_runtime_env';
    memlocation := GetProcedureAddress(h_axrt, funname);
    if assigned(memlocation) then  pointer(__set_runtime_env) := memlocation else writeln('unable to locate function ' + funname);

    funname := '__kick_start';
    memlocation := GetProcedureAddress(h_axrt, funname);
    if assigned(memlocation) then  pointer(__kick_start) := memlocation else writeln('unable to locate function ' + funname);

    // assume everything went ok.

    writeln('set runtime environment');
    __set_runtime_env(RT_VER);


    writeln('kick start startup entry');
    __kick_start(@__startup_entry, RT_VER);


    // #######################

    if unloadlibrary(h_axrt)
      then writeln('unloaded ax runtime library')
    else writeln('failed unloading ax runtime library');
  end
  else writeln('failed to load ax runtime library');

  writeln('end');
end.

Which translates into the following assembler source-code (debug enabled, again note that the line numbers might differ a little):

Code: [Select]

Assembler source code attached inside zipfile because the board is limited to 20000 characters per post).


And then running it with AXRT, the output from the terminal becomes:

Code: [Select]

begin
loaded ax runtime library
retrieving libaxrt function pointers
set runtime environment
<<INFO>>: AxRT 41.1
<<INFO>>: Using absolute paths.
<<INFO>>: AXRT_ROOT environment variable set
<<INFO>>: RUNTIME_ROOT: /home/magorium/bin/AXRT_411/
<<INFO>>: AXRTSYS     : ROOT:home/magorium/bin/AXRT_411/
<<INFO>>: USERSYS     : ROOT:home/magorium/.axrt/
kick start startup entry
libaxrt-4.0.so: cannot open shared object file: No such file or directory
<<INFO>>: CURRENT_DIR : ROOT:home/magorium/bin/AXRT_411/Work/fpc/
<<INFO>>: PROGRAM DIR : ROOT:home/magorium/bin/AXRT_411/Work/fpc/
<<INFO>>: PROGRAM NAME: linuxaxrttest
hello axrt from fpc - still linux
before AvailMem
Runtime error 216 at $000000000040121D
  $000000000040121D  __STARTUP_ENTRY,  line 60 of linuxaxrttest.pas
  $00007FF2C33DDB91 line 225 of /media/ramdisk/repo411/AROS/arch/all-runtime/axrt/startup/startup_lib.c
  $00007FF2C224C693 line 85 of /media/ramdisk/repo411/AROS/rom/dos/./boot.c
  $00007FF2C222FBFD line 134 of /media/ramdisk/repo411/AROS/rom/dos/./exit.c
  $00007FF2C222A384 line 739 of /media/ramdisk/repo411/AROS/rom/dos/./createnewproc.c
  $00007FF2C29ADFEB line 308 of /media/ramdisk/repo411/AROS/rom/exec/./newaddtask.c
  $00007FF2C33A0FA3


AvailMem(0) =


running it with gdb shows:

Code: [Select]

Starting program: /home/magorium/bin/AXRT_411/Work/fpc/linuxaxrttest
begin
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
loaded ax runtime library
retrieving libaxrt function pointers
set runtime environment
<<INFO>>: AxRT 41.1
<<INFO>>: Using absolute paths.
<<INFO>>: AXRT_ROOT environment variable set
<<INFO>>: RUNTIME_ROOT: /home/magorium/bin/AXRT_411/
<<INFO>>: AXRTSYS     : ROOT:home/magorium/bin/AXRT_411/
<<INFO>>: USERSYS     : ROOT:home/magorium/.axrt/
kick start startup entry
[New Thread 0x7ffff7bc1700 (LWP 30833)]
[New Thread 0x7ffff6b3d700 (LWP 30834)]
[New Thread 0x7ffff633c700 (LWP 30835)]
[New Thread 0x7ffff5b3b700 (LWP 30836)]
[New Thread 0x7ffff533a700 (LWP 30837)]
[New Thread 0x7ffff4b39700 (LWP 30838)]
[New Thread 0x7fffe7fff700 (LWP 30839)]
[New Thread 0x7fffe71e0700 (LWP 30840)]
[New Thread 0x7fffe651a700 (LWP 30841)]
[New Thread 0x7fffe5d19700 (LWP 30842)]
[New Thread 0x7fffe5518700 (LWP 30843)]
[New Thread 0x7fffe4d17700 (LWP 30844)]
[Thread 0x7ffff7bc1700 (LWP 30833) exited]
libaxrt-4.0.so: cannot open shared object file: No such file or directory
[New Thread 0x7ffff7bc1700 (LWP 30845)]
[Thread 0x7fffe5d19700 (LWP 30842) exited]
<<INFO>>: CURRENT_DIR : ROOT:home/magorium/bin/AXRT_411/Work/fpc/
<<INFO>>: PROGRAM DIR : ROOT:home/magorium/bin/AXRT_411/Work/fpc/
<<INFO>>: PROGRAM NAME: linuxaxrttest
[New Thread 0x7fffcbfff700 (LWP 30846)]
hello axrt from fpc - still linux
before AvailMem


Thread 14 "New Process" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff7bc1700 (LWP 30845)]
0x000000000040121d in __STARTUP_ENTRY (ARGSTR=0x0, ARGSIZE=0, SYSBASE=0x7ffff0001780)
    at linuxaxrttest.pas:60
60        writeln('AvailMem(0) = ', Availmem(0));


The gdb backtrace:

Code: [Select]

#0  0x000000000040121d in __STARTUP_ENTRY (ARGSTR=0x0, ARGSIZE=0, SYSBASE=0x7ffff0001780)
    at linuxaxrttest.pas:60
#1  0x00007ffff7dcfb91 in RunProgram (sysbase=0x7ffff0001780, _m=0x7fffffffced0)
    at /media/ramdisk/repo411/AROS/arch/all-runtime/axrt/startup/startup_lib.c:225
#2  0x00007ffff6c3e693 in __program_trampoline () at /media/ramdisk/repo411/AROS/rom/dos/./boot.c:85
#3  0x00007ffff6c21bfd in CallEntry (argptr=0x0, argsize=0,
    entry=0x7ffff6c3e58e <__program_trampoline>, me=0x7fffcc017aa0)
    at /media/ramdisk/repo411/AROS/rom/dos/./exit.c:134
#4  0x00007ffff6c1c384 in DosEntry () at /media/ramdisk/repo411/AROS/rom/dos/./createnewproc.c:739
#5  0x00007ffff739ffeb in __pthread_trampoline (ptr=0x7fffcc016c70)
    at /media/ramdisk/repo411/AROS/rom/exec/./newaddtask.c:308
#6  0x00007ffff7d92fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#7  0x00007ffff7edd4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

See also attached zip file which includes all the relevant information.

posted in stages because the stupid board software removes all perfectly fine and accepted text whenever you post a single line that isn't supposedly valid so then need to start your whole post all over again, in the hopes you are able to identify that one single line that is suppose to be invalid.


If everything went ok then this post is done now.


edit: I have looked at the asm code a little longer but i am still unable to see the (probably) obvious. I also made a better test, not combining pascal rtl calls with axrt ABI calls and it crashes at assigning the return-value from a syscall to availmem to a ULONG variable (and not writing the result value directly). A.t.m. i have no idea how to continue forward from this point. Do you perhaps have any idea deadw00d (or someone else for that matter).

[magorium]

I have a general question about FCP on x86_64 (Linux I guess). Which code model (small, medium, large) is the default for the FPC compiler?

afaik there is no provision for selecting different code models (for 64 bit targets) so i am therefor (very ignorantly) inclined to say that there is no default 

I have no idea about the underlying reason why you asked, so is there need for you for a more precise/exact answer ?

[magorium]

Isn't there a way to pre-compile the "proxy" units (those who forward calls from Pascal to AROS) on AROS x86_64

Yes, it is possible to pre-compile separate units for a specific target. However, precompiled units for different targets can not be mixed and matched together by the FPC compiler itself.

.. and the use these pre-compiled units with normal Linux FPC?

Use, is a simple word for something that complicated 

It is complicated because how to mix the two abi's together when separated because of compiler target is beyond my knowledge. Yes, i can create object files and mix them up with f.e. objects created by a c-compiler, but then i would know this is happening upfront so that i can account for situations inside the code that requires preparations, such as for example the cdecl definition (because that is how c expects the arguments to be passed along).

Thinking ahead, that  would also imply that there is need for multi compilation (not a deal-breaker perse) which requires additional work as FPC has no provision for that.

Besides that, the FPC startup code for both targets differ, which might perhaps be my current culprit in that produced code crashes. I have no idea hwo that impact things.

Last item is the required linking voo-doo that would be required to mash up the generated code .... did i mention i am not very proficient with ld ?

The above might sound as excuses, but is not meant as such. Just trying to look ahead and see what bridges need to be crossed. I figured that attempting to add the new axrt syscall ABI would be the shorter/simpler/easier road to take.

The only addition in AROS ABIv11 is that library base is placed into R12 register before making a call. Rest is normal SysV. In case of C this addition is handled for example in a linker library (= pre-compiled unit).

Yes, i understand. The new ABI is now added to the compiler. However, i am not sure i have added it correctly so would have to verify that with someone like ALB, Chain-Q or someone else more proficient with asm. Perhaps you yourself have an idea or two about what might be wrong ?

[deadwood]

@magorium

Can you share compiled Pascal binary? I will debug it locally. Please make the function as simple as possible:

one writeln call
call to AvailMem and assignment to variable
second writeln call

[deadwood]

I have a general question about FCP on x86_64 (Linux I guess). Which code model (small, medium, large) is the default for the FPC compiler?

afaik there is no provision for selecting different code models (for 64 bit targets) so i am therefor (very ignorantly) inclined to say that there is no default 

I have no idea about the underlying reason why you asked, so is there need for you for a more precise/exact answer ?

Sorry for not sharing this. It is more related to AROS than to AxRuntime. AROS x86_64 assumes that large code model is used. Is small model is used, binaries will still work if user has less than 2GB of RAM. If user has more, then binaries will randomly crash depending on whether they were loaded below or above 2GB and there is no AROS-side control of where this happens.

[deadwood]

I noticed one thing in assembler:

Code: [Select]

call *-36(%r12)

AvailMem LVO is 36, but the offset in bytes is -36*8. Maybe that's the problem?

Edit:

That's how this

Code: [Select]

#include <proto/exec.h>

int main()
{
    AvailMem(0);
}

looks in assembler

Code: [Select]

0000000000000000 <main>:
   0: 53                    push   %rbx
   1: 48 a1 00 00 00 00 00 movabs 0x0,%rax
   8: 00 00 00
   b: 4c 89 e3              mov    %r12,%rbx
   e: 49 89 c4              mov    %rax,%r12
  11: 31 ff                xor    %edi,%edi
  13: ff 90 e0 fe ff ff    callq  *-0x120(%rax)
  19: 49 89 dc              mov    %rbx,%r12
  1c: 31 c0                xor    %eax,%eax
  1e: 5b                    pop    %rbx
  1f: c3                    retq   


Offset is 0x120 => 288 => 36*8

[magorium]

@magorium

Can you share compiled Pascal binary? I will debug it locally. Please make the function as simple as possible:

one writeln call
call to AvailMem and assignment to variable
second writeln call

Although you also already responded on another matter, i have uploaded a source/asm and binary here: https://file.io/BJbxnduLFyVo


Please let me know in case the link doesn't work for you (as i just recently used this website).


TIA

[AMIGASYSTEM]

Not working "The file you requested has been deleted"

[magorium]

I noticed one thing in assembler:

Code: [Select]

   call   *-36(%r12)

AvailMem LVO is 36, but the offset in bytes is -36*8. Maybe that's the problem?

Thank you !!. U nailed it 

Preliminary test shows this is indeed the culprit. I'll mock up a more enhanced test example tomorrow and check that. I overlooked the offsets literally dozens of times 

I still need to figure out how to fix that inside the compiler source-code but for now i just multiplied the offset by 8 

Thanks again.

[magorium]

Not working "The file you requested has been deleted"

Hi AMIGASYSTEM.

Thanks you for letting me know. However, do notice that i use the free plan. That means that the file can only be downloaded once. If you were sooner than the one who downloaded the file before you then you would not have received that message.

That was reason for me to ask (deadw00d) if it worked for him. If not then i would need to upload the file again.

[AMIGASYSTEM]

Why don't you use Google Drive, that way you can add or delete files as many times as you want, Google Drive offers 15 GB free, if you have more emails you can have more Google drive

[deadwood]

Sorry for not sharing this. It is more related to AROS than to AxRuntime. AROS x86_64 assumes that large code model is used. Is small model is used, binaries will still work if user has less than 2GB of RAM. If user has more, then binaries will randomly crash depending on whether they were loaded below or above 2GB and there is no AROS-side control of where this happens.

I think eventually I will add proper support for small model executables, so even if FPC creates them now, it should not be a problem for the future.