table of contents
PCREJIT(3) | Library Functions Manual | PCREJIT(3) |
NAME¶
PCRE - Perl-compatible regular expressionsPCRE JUST-IN-TIME COMPILER SUPPORT¶
Just-in-time compiling is a heavyweight optimization that can greatly speed up pattern matching. However, it comes at the cost of extra processing before the match is performed. Therefore, it is of most benefit when the same pattern is going to be matched many times. This does not necessarily mean many calls of a matching function; if the pattern is not anchored, matching attempts may take place many times at various positions in the subject, even for a single call. Therefore, if the subject string is very long, it may still pay to use JIT for one-off matches. JIT support applies only to the traditional Perl-compatible matching function. It does not apply when the DFA matching function is being used. The code for this support was written by Zoltan Herczeg.8-BIT and 16-BIT SUPPORT¶
JIT support is available for both the 8-bit and 16-bit PCRE libraries. To keep this documentation simple, only the 8-bit interface is described in what follows. If you are using the 16-bit library, substitute the 16-bit functions and 16-bit structures (for example, pcre16_jit_stack instead of pcre_jit_stack).AVAILABILITY OF JIT SUPPORT¶
JIT support is an optional feature of PCRE. The "configure" option --enable-jit (or equivalent CMake option) must be set when PCRE is built if you want to use JIT. The support is limited to the following hardware platforms:ARM v5, v7, and Thumb2
Intel x86 32-bit and 64-bit
MIPS 32-bit
Power PC 32-bit and 64-bit
SIMPLE USE OF JIT¶
You have to do two things to make use of the JIT support in the simplest way:(1) Call pcre_study() with the PCRE_STUDY_JIT_COMPILE option for
each compiled pattern, and pass the resulting pcre_extra block to
pcre_exec().
(2) Use pcre_free_study() to free the pcre_extra block when it is
no longer needed instead of just freeing it yourself. This
ensures that any JIT data is also freed.
#ifndef PCRE_STUDY_JIT_COMPILE
#define PCRE_STUDY_JIT_COMPILE 0
#endif
#ifdef PCRE_CONFIG_JIT
pcre_free_study(study_ptr);
#else
pcre_free(study_ptr);
#endif
UNSUPPORTED OPTIONS AND PATTERN ITEMS¶
The only pcre_exec() options that are supported for JIT execution are PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, and PCRE_NOTEMPTY_ATSTART. Note in particular that partial matching is not supported. The unsupported pattern items are:\C match a single byte; not supported in UTF-8 mode
(?Cn) callouts
(*COMMIT) )
(*MARK) )
(*PRUNE) ) the backtracking control verbs
(*SKIP) )
(*THEN) )
RETURN VALUES FROM JIT EXECUTION¶
When a pattern is matched using JIT execution, the return values are the same as those given by the interpretive pcre_exec() code, with the addition of one new error code: PCRE_ERROR_JIT_STACKLIMIT. This means that the memory used for the JIT stack was insufficient. See "Controlling the JIT stack" below for a discussion of JIT stack usage. For compatibility with the interpretive pcre_exec() code, no more than two-thirds of the ovector argument is used for passing back captured substrings. The error code PCRE_ERROR_MATCHLIMIT is returned by the JIT code if searching a very large pattern tree goes on for too long, as it is in the same circumstance when JIT is not used, but the details of exactly what is counted are not the same. The PCRE_ERROR_RECURSIONLIMIT error code is never returned by JIT execution.SAVING AND RESTORING COMPILED PATTERNS¶
The code that is generated by the JIT compiler is architecture-specific, and is also position dependent. For those reasons it cannot be saved (in a file or database) and restored later like the bytecode and other data of a compiled pattern. Saving and restoring compiled patterns is not something many people do. More detail about this facility is given in the pcreprecompile documentation. It should be possible to run pcre_study() on a saved and restored pattern, and thereby recreate the JIT data, but because JIT compilation uses significant resources, it is probably not worth doing this; you might as well recompile the original pattern.CONTROLLING THE JIT STACK¶
When the compiled JIT code runs, it needs a block of memory to use as a stack. By default, it uses 32K on the machine stack. However, some large or complicated patterns need more than this. The error PCRE_ERROR_JIT_STACKLIMIT is given when there is not enough stack. Three functions are provided for managing blocks of memory for use as JIT stacks. There is further discussion about the use of JIT stacks in the section entitled "JIT stack FAQ" below. The pcre_jit_stack_alloc() function creates a JIT stack. Its arguments are a starting size and a maximum size, and it returns a pointer to an opaque structure of type pcre_jit_stack, or NULL if there is an error. The pcre_jit_stack_free() function can be used to free a stack that is no longer needed. (For the technically minded: the address space is allocated by mmap or VirtualAlloc.) JIT uses far less memory for recursion than the interpretive code, and a maximum stack size of 512K to 1M should be more than enough for any pattern. The pcre_assign_jit_stack() function specifies which stack JIT code should use. Its arguments are as follows:pcre_extra *extra
pcre_jit_callback callback
void *data
(1) If callback is NULL and data is NULL, an internal 32K block
on the machine stack is used.
(2) If callback is NULL and data is not NULL, data must be
a valid JIT stack, the result of calling pcre_jit_stack_alloc().
(3) If callback not NULL, it must point to a function that is called
with data as an argument at the start of matching, in order to
set up a JIT stack. If the result is NULL, the internal 32K stack
is used; otherwise the return value must be a valid JIT stack,
the result of calling pcre_jit_stack_alloc().
During thread initalization
thread_local_var = pcre_jit_stack_alloc(...)
During thread exit
pcre_jit_stack_free(thread_local_var)
Use a one-line callback function
return thread_local_var
JIT STACK FAQ¶
(1) Why do we need JIT stacks?EXAMPLE CODE¶
This is a single-threaded example that specifies a JIT stack without using a callback.int rc;
int ovector[30];
pcre *re;
pcre_extra *extra;
pcre_jit_stack *jit_stack;
re = pcre_compile(pattern, 0, &error, &erroffset, NULL);
/* Check for errors */
extra = pcre_study(re, PCRE_STUDY_JIT_COMPILE, &error);
jit_stack = pcre_jit_stack_alloc(32*1024, 512*1024);
/* Check for error (NULL) */
pcre_assign_jit_stack(extra, NULL, jit_stack);
rc = pcre_exec(re, extra, subject, length, 0, 0, ovector, 30);
/* Check results */
pcre_free(re);
pcre_free_study(extra);
pcre_jit_stack_free(jit_stack);
SEE ALSO¶
pcreapi(3)AUTHOR¶
Philip Hazel (FAQ by Zoltan Herczeg) University Computing Service Cambridge CB2 3QH, England.
REVISION¶
Last updated: 08 January 2012 Copyright (c) 1997-2012 University of Cambridge.