The new compiler is complete!

Here you can post threads on the development of the current release of the core (100)

Moderator: POL Developer

Post Reply
Syzygy
POL Developer
Posts: 7
Joined: Fri Apr 17, 2020 10:02 pm

The new compiler is complete!

Post by Syzygy » Tue Sep 15, 2020 8:35 pm

Hi everyone! I wanted to announce here that the new eScript compiler is complete.

It’s available on the master branch and in nightly builds.

Please be sure to run ecompile in comparison mode, as described in core-changes, in order to ensure that the new compiler doesn’t change the behavior of any of your scripts.

The original compiler is still available. I’ll delete it only after both compilers have been available in an official release for at least several months.

I also wanted to mention some of the reasons why I rewrote the compiler:
  • I began work on the original compiler when I was 20 years old! It’s an embarrassing mess. It’s hard to work with. I imagine it’s nearly impossible for anyone but me to understand. It’s difficult to add anything new or change anything. The new compiler is composed of manageable pieces and is a lot easier to understand.
  • As a project, the new compiler was safe because it is testable. In the time that I haven’t been working on POL, the good people maintaining it have extended the test suite. That decreases risk. I’ve been away from the codebase for a long time, so this was a nice way to start to re-familiarize myself with it.
  • Because the new compiler produces output that is identical* to that of the original compiler, it's easy to know that it didn't break any scripts. This has been tested against the test suite, ModernDistro, and ClassicDistro.
    • *with a few exceptions, like changed operator precedence, but ecompile’s “comparison mode” will find any scripts that changed. Notably, the operator precedence changes did not change the .ecl output of any script in ModernDistro or ClassicDistro. The changes to operator precedence are intended to make it so more expressions do what you want without parentheses, rather than having to add them.
  • It’s a lot easier to see what is in the instructions that the compiler emits for the executor, because the compiler creates each instruction right before emitting it.
  • We can improve eScript and the compiler much more easily. For example:
    • add a ternary operator
    • add an “is” operator as an alternative to TypeOfInt
    • warn about expressions that are likely to be in error
    • better expression optimizations
    • optimize the instruction set (generally by reducing the number of instructions required to do some common things)
    • add short-circuit logical operators
      • this was possible with the original compiler, but tricky. I did it with the “Elvis” operator, which is similar to short-circuit OR, but it had to be done in the optimizer. Compare:
        • PR-137 adding the Elvis operator to the original compiler (relevant code)
        • PR-293 adding the Elvis operator to the new compiler
  • The new compiler shares far fewer datatypes with the executor, so we’ll be able to simplify some of the executor’s datatypes after deleting the old compiler.

Yukiko
Distro Developer
Posts: 2773
Joined: Thu Feb 02, 2006 1:41 pm
Location: San Antonio, Texas
Contact:

Re: The new compiler is complete!

Post by Yukiko » Tue Sep 15, 2020 8:58 pm

For those who may not be aware, Syzygy is the creator of the Penultimate OnLine emulator, POL.

Thank you Syzygy for the new compiler.

ThisIsMe
Distro Developer
Posts: 97
Joined: Sun Jul 17, 2016 1:29 am
Contact:

Re: The new compiler is complete!

Post by ThisIsMe » Tue Sep 15, 2020 9:23 pm

Curious, whats a Short-Circuit Logical expression?

Syzygy
POL Developer
Posts: 7
Joined: Fri Apr 17, 2020 10:02 pm

Re: The new compiler is complete!

Post by Syzygy » Tue Sep 15, 2020 9:46 pm

Given this example code

Code: Select all

    if(( item.IsA( POLCLASS_CONTAINER ) || item.IsA( POLCLASS_DOOR )) && item.locked )
      SendSysMessage( mobile, "That item is locked!" );
      return 0;
    endif
With short-circuit logical evaluation, if

Code: Select all

item.IsA( POLCLASS_CONTAINER ) 
is true, then

Code: Select all

item.IsA( POLCLASS_DOOR )
would not need to be evaluated.

If both of those are false (if the left-hand side of the && condition is false), then item.locked would not need to be evaluated.

User avatar
Pumpkins
Apprentice Poster
Posts: 52
Joined: Mon Jan 22, 2007 6:06 am

Re: The new compiler is complete!

Post by Pumpkins » Mon Oct 19, 2020 12:48 am

Hello Syzygy,

I want to thank you for reworking the ecompile, I'm really enjoying it. It founds a lot of errors the old compile never did. Like faulty functions that are included but not referenced, and it is helping me find functions that are not being used at all.

I have two questions;

About the numbers the new compile prints when an error occurs:
C:\UO\Distro\scripts\include\itemutil.inc:27:34: error: Deprecated '=' found: did you mean '==' or ':='?
The number 27 means the line, but what does 34 means?

I'm running the new compile in comparison mode and I got an abort but the new compiler doesn't tell me where the mismatch occurred, only:
Execution aborted due to: Compiler output mismatch
Im for sure missing something here, how can I see the script causing the abort?

DevGIB
Grandmaster Poster
Posts: 247
Joined: Mon Feb 06, 2006 6:12 am

Re: The new compiler is complete!

Post by DevGIB » Mon Oct 19, 2020 6:14 am

The number 27 means the line, but what does 34 means?
I might be mistaken, but I think that the 34 is the column or character in that line that I is taking issue with.

User avatar
Pumpkins
Apprentice Poster
Posts: 52
Joined: Mon Jan 22, 2007 6:06 am

Re: The new compiler is complete!

Post by Pumpkins » Mon Oct 19, 2020 10:40 am

Yes, I made some testes and it is like you said the column. Nice new feature! :)

About my 2. question, I found what script is causing the abort inside ecompile.log
\guilds\scripts\guilds.src: 0 errors, 0 warnings.
Listings match: 0
Binary (.ecl) outputs match: 0
But I don't get any error at all, just the abort telling me that it doesn't match. o.O

It generates 4 new files inside Pols main folder:
new-compiler.ecl
og-compiler.ecl
new-compiler.lst
og-compiler.lst

If I compare both .lst I see some differences but I can't tell what is causing it.

Syzygy
POL Developer
Posts: 7
Joined: Fri Apr 17, 2020 10:02 pm

Re: The new compiler is complete!

Post by Syzygy » Mon Oct 19, 2020 10:02 pm

It might help to:
1. Look at the differences between og_compiler.lst and new_compiler.lst
2. Look in og_compiler.lst to see what the source code is at the difference (it includes a bit of the source code, and the line number)
3. Post here some of the source near that, and some of the .lst files from where they are the same to where they are different.

It takes some troubleshooting / deduction to tell what is causing the difference, but it is impossible to do that without knowing anything about what is in the different .lst files (therefore the different .ecl files) and what is in the source code near to that point.

Post Reply