| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This lexer is being written as a chunklet, not quite quite complete yet
as it lacks some functionality to make it generally useful for things,
but good enough for the gluegen use case now. So it's in the repo now
and we can go ahead and use it for this instead of having this
hacked-to-pieces third party thing that's nowhere near as efficient. In
the future the goal is to have a decently usable library for any sort of
C metaprogramming needs, not just within this project.
This design does the data-oriented thing of storing as little about each
token as possible (just 5 bytes), and re-lexing specific pieces only
when necessary. In some cases, full validation of correct syntax is
only possible through this secondary step. Since the use case is
metaprogramming and code-generation rather than the reimplementation of
Clang, this slight sloppiness in validation doesn't seem too bad.
It's been a while since I did a proper performance measurement of this
code but in some crude tests I did in the past the primary tokenisation
step was running at well over 500MB/s, which is fast enough for me. It's
possible that this version is a tiny bit slower due to the added
complexity of making UCNs work. UCNs, incidentally, are one of the
dumbest features of C by far. However, unlike trigraphs - which this
lexer does not handle - UCNs are still in the language, so it's kind of
sort of necessary to support them.
It's probably still possible to come up with a faster design with some
SIMD trickery, but the main loop is currently branchless and the lookup
tables are too big for the SIMD lookup things, so that seems kind of
hard. A separate SIMD path for whitespace or comment runs seems dubious
as it would introduce branch mispredictions everywhere. I also made
previous attempts to unroll the main loop and every attempt just made it
slower, so I guess code size is a significant factor.
Optimising the secondary tokenisation of identifiers (and later numerals
and string/character literals once those are handled) is still on the
cards, but since that happens less often, I don't know how much
difference it'll make.
At any rate, in a multithreaded context this thing would already come
pretty close to SSD speeds, if open-read-close syscall overhead doesn't
get in the way first. I imagine it's fast enough for anyone who hasn't
*already* written something faster.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- As much as possible avoid dragging system headers into translation
units. This should avoid namespace pollution and, hopefully, speed up
builds a little bit.
- Avoid leaning on the UCRT so much on Windows - prefer native win32
calls and native file handles except where doing so is inconvenient
(in particular, for stat(), which we might try and replace later).
- Also, switch from SystemFunction036 to ProcessPrng on Windows. This
requires us to generate a stub for bcryptprimitives.dll because
Microsoft haven't bothered to provide a link library, but the function
is better-documented and seems to be a more direct under-the-hood call
as well. Apparently it's what's used by the major web browsers these
days, which seems like a good indication it's stable and trusted.
- Lastly, remove a bunch of functions and macros and stuff that weren't
actually being used. It seems good to try and keep the scope of
OS-dependent stuff relatively contained and only add to it when
actually required.
|
| |
|
|
|
|
|
|
|
| |
That sorting function was a bit wonky, so make it just a little bit
wonky instead. chibicc would produce confusing lex errors if given a
stray single quote somewhere, so make it give non-confusing errors.
Also get rid of canonicalize_newline() because it's unnecessary for SST
so long as Windows Git isn't left in its default misconfigured state.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Another big one. Here's a list of things:
- Since the upcoming C23 standardises typeof(), use it as an extension
for the time being in order to allow passing arbitrary types as
macro/codegen parameters. It wouldn't have been a big leap to do this
even without standardisation since it's apparently an easy extension
to implement - and also, to be honest, this project is essentially glued
to Clang anyway so who cares.
- Likewise, bool, true and false are becoming pre-defined, so
pre-pre-define them now in order to get the benefit of not having to
remember one header everywhere.
- Really ungodly/amazing vcall macro stuff now allows us to call C++
virtual functions like regular C functions. It's pretty cool!
- Events can now take arbitrary parameters and come in two types:
regular events and predicates.
All this makes the base code even uglier but makes the feature
implementation nicer. In other words, it places more of the cognitive
burden on myself and less on other people who might want to contribute.
This is a good tradeoff, because I'm a genius.
|
|
|
With code from Bill. Thanks Bill!
|
Michael Smith
