hxuanyu/intellij-sdk-code-samples
1
0
Fork 0
mirror of https://github.com/JetBrains/intellij-sdk-code-samples.git synced 2025-07-28 01:07:49 +08:00
Code Issues Packages Projects Releases Wiki Activity

[site] external_builder_api: formatting, wording

Browse Source
This commit is contained in:
Yann Cébron 2015-03-19 11:57:42 +01:00
parent 05261217f0
commit 77cde95ef4
1 changed files with 28 additions and 28 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

56
external_builder_api.md
View File

@ -12,37 +12,37 @@ INITIAL_SOURCE https://confluence.jetbrains.com/display/IDEADEV/External+Builder
When the user invokes an action that involves executing an external build (Make, Build Artifacts, etc.), the following steps happen: When the user invokes an action that involves executing an external build (Make, Build Artifacts, etc.), the following steps happen:
* Before-compile tasks are executed in the IntelliJ process. * Before-compile tasks are executed in the IDE process.
* Some source generation tasks that depend on the PSI (e.g. UI designer form to source compilation) are executed in the IntelliJ process. * Some source generation tasks that depend on the PSI (e.g. UI designer form to source compilation) are executed in the IDE process.
* BuildTargetScopeProvider extensions are called to calculate the scope of the external build (the set of build targets to compile based on the target module to make and the known set of changes). * `BuildTargetScopeProvider` extensions are called to calculate the scope of the external build (the set of build targets to compile based on the target module to make and the known set of changes).
* The external build process is spawned (or an existing build process background process is reused). * The external build process is spawned (or an existing build process background process is reused).
* The external build process loads the IntelliJ IDEA project model (.idea, .iml files and so on), represented by a ```JpsModel``` instance. * The external build process loads the project model (.idea, .iml files and so on), represented by a ```JpsModel``` instance.
* The full tree of targets to build is calculated, based on the dependencies of each build target to be compiled. * The full tree of targets to build is calculated, based on the dependencies of each build target to be compiled.
* For each target, the set of builders capable of building this target is calculated. * For each target, the set of builders capable of building this target is calculated.
* For every target and every builder, the build() method is called. This can happen in parallel if the "Compile independent modules in parallel" option is enabled in the settings. For module-level builders, the order of invoking builders for a single target is determined by their category; for other builders, the order is undefined. * For every target and every builder, the `build()` method is called. This can happen in parallel if the "Compile independent modules in parallel" option is enabled in the settings. For module-level builders, the order of invoking builders for a single target is determined by their category; for other builders, the order is undefined.
* Caches to record the state of the compilation are saved. * Caches to record the state of the compilation are saved.
* Compilation messages reported through the CompileContext API are transmitted to the IntelliJ process and displayed in the UI (in the Messages view). * Compilation messages reported through the `CompileContext` API are transmitted to the IDE process and displayed in the UI (in the *Messages* view).
* Post-compile tasks are executed in the IntelliJ process. * Post-compile tasks are executed in the IDE process.
### Incremental Build ### Incremental Build
To support incremental build, the build process uses a number of caches which are persisted between build invocations. Even if your compiler doesn't support incremental build, you still need to report correct information so that incremental build works correctly for other compilers. To support incremental build, the build process uses a number of caches which are persisted between build invocations. Even if your compiler doesn't support incremental build, you still need to report correct information so that incremental build works correctly for other compilers.
* ```SourceToOutputMapping``` is a many-to-many relationship between source files and output files ("which source files were used to produce the specified output file"). It's filled by calls to BuildOutputConsumer.registerOutputFile() and ModuleLevelBuilder.OutputConsumer.registerOutputFile(). * ```SourceToOutputMapping``` is a many-to-many relationship between source files and output files ("which source files were used to produce the specified output file"). It's filled by calls to `BuildOutputConsumer.registerOutputFile()` and `ModuleLevelBuilder.OutputConsumer.registerOutputFile()`.
* ```Timestamps``` records the timestamp of each source file when it was compiled by each build target. (If the compilation was invoked multiple times with different scopes and the file was changed inbetween, the last compiled timestamps for different build targets may vary.) It's updated automatically by JPS. * ```Timestamps``` records the timestamp of each source file when it was compiled by each build target. (If the compilation was invoked multiple times with different scopes and the file was changed in the meantime, the last compiled timestamps for different build targets may vary.) It's updated automatically by JPS.
IntelliJ monitors the changes of the project content and uses the information from those caches to generate the set of dirty and deleted files for every compilation. (Dirty files need to be recompiled, and deleted files need to have their output deleted). A builder can also report additional files as dirty (e.g. if a method is deleted, the builder can report the classes using this method as dirty.) A module-level builder can add some files to the dirty scope; if this happens, and if the builder returns ```ADDITIONAL_PASS_REQUIRED``` from its build() method, another round of builder execution for the same module chunk will be started with the new dirty scope. The IDE monitors the changes of the project content and uses the information from those caches to generate the set of dirty and deleted files for every compilation. (Dirty files need to be recompiled, and deleted files need to have their output deleted). A builder can also report additional files as dirty (e.g. if a method is deleted, the builder can report the classes using this method as dirty.) A module-level builder can add some files to the dirty scope; if this happens, and if the builder returns ```ADDITIONAL_PASS_REQUIRED``` from its `build()` method, another round of builder execution for the same module chunk will be started with the new dirty scope.
A builder may also want to have its custom caches to store additional information to support partial recompilation of a target (e.g. the dependencies between Java files in a module). To store this data, you can either store arbitrary files in the directory returned from ```BuildDataManager.getDataPaths().getTargetDataRoot()``` or use a higher-level API: ```BuildDataManager.getStorage()``` A builder may also want to have its custom caches to store additional information to support partial recompilation of a target (e.g. the dependencies between Java files in a module). To store this data, you can either store arbitrary files in the directory returned from ```BuildDataManager.getDataPaths().getTargetDataRoot()``` or use a higher-level API: ```BuildDataManager.getStorage()```
@ -51,39 +51,39 @@ To pass custom data between the invocation of the same builder between multiple
### Services and extensions in External Builder ### Services and extensions in External Builder
The external builder process uses the standard Java The external builder process uses the standard Java
[services](http://docs.oracle.com/javase/6/docs/api/java/util/ServiceLoader.html) [services](http://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html)
mechanism to support plugins. There are several service interfaces (e.g. BuilderService) which can be implemented in plugins to extend the builder functionality. An implementation of a service need to be registered by creating META-INF/services/<service-interface-fqn> file containing the qualified name of the implementation class. E.g. BuilderService's implementations are registered in META-INF/services/org.jetbrains.jps.incremental.BuilderService file. These files don't have extensions so you need to map corresponding patterns to text files in IDEA settings. mechanism to support plugins. There are several service interfaces (e.g. `BuilderService`) which can be implemented in plugins to extend the builder functionality. An implementation of a service need to be registered by creating `META-INF/services/<service-interface-fqn>` file containing the qualified name of the implementation class. E.g. `BuilderService` implementations are registered in `META-INF/services/org.jetbrains.jps.incremental.BuilderService` file. These files don't have extensions so you need to map corresponding patterns to text files in IDE settings.
### Registering a plugin for External Builder ### Registering a plugin for External Builder
Sources of a plugin for External Builder should be put in a separate module. By convention such module has name '...-jps-plugin' and its sources are placed under 'jps-plugin' directory in the main plugin directory. Use compileServer.plugin extension to add the plugin to classpath of external build process, the plugin jar should be named "<jps module name>.jar". 'Build' | 'Prepare Plugin Module for deployment' action will automatically pack 'jps-plugin' part to a separate jar accordingly. Sources of a plugin for External Builder should be put in a separate module. By convention such module has name '...-jps-plugin' and its sources are placed under 'jps-plugin' directory in the main plugin directory. Use `<compileServer.plugin>` extension to add the plugin to classpath of external build process, the plugin jar should be named "<jps module name>.jar". 'Build' | 'Prepare Plugin Module for deployment' action will automatically pack 'jps-plugin' part to a separate jar accordingly.
### Debugging a plugin for External Builder ### Debugging a plugin for External Builder
Start IDEA with your plugin with the following VM option Start IDE with your plugin with the following VM option
``` ```
-Dcompiler.process.debug.port=<port-number> -Dcompiler.process.debug.port=<port-number>
``` ```
After that every time compilation is run in the test IDEA, the build process will wait for debugger connection at this port and only then proceed. In working copy of IDEA a "Remote" run configuration should be created and pointed it to this port. Specifying port "-1" will disable debugging mode. After that every time compilation is run in the test IDE, the build process will wait for debugger connection on this port and only then proceed. In working copy of IDE a "Remote" run configuration should be created and pointed to this port. Specifying port "-1" will disable debugging mode.
### Profiling external build process ### Profiling external build process
The build process has built-in self-cpu-profiling capabilities. To enable them do the following: The build process has built-in self-cpu-profiling capabilities. To enable them do the following:
1. Copy <idea-home>/lib/yjp-controller-api-redist.jar and <idea-home>/bin/yjpagent.\* files to <idea-system-dir>/compile-server 1. Copy `<ide-home>/lib/yjp-controller-api-redist.jar` and `<ide-home>/bin/yjpagent.\*` files to `<ide-system-dir>/compile-server`
2. In "Settings \| Compiler \| Additional compiler process VM options" field enter \-Dprofiling.mode=true 2. In "Settings \| Compiler \| Additional compiler process VM options" field enter `-Dprofiling.mode=true`
3. Make sure automatic make is turned off 3. Make sure automatic make is turned off
After this every build process run should result in a CPU snapshot stored in <user-home>/Snapshots directory. After this every build process run should result in a CPU snapshot stored in <user-home>/Snapshots directory.
Snapshots are named like "ExternalBuild\-\{date\}.snapshot". Snapshots are named like "ExternalBuild\-\{date\}.snapshot".
Specifying \-Dprofiling.mode=false will turn profiling off. Specifying `-Dprofiling.mode=false` will turn profiling off.
Please capture a couple of snapshots for the situations in which you believe the build should work much faster than it works. Please capture a couple of snapshots for the situations in which you believe the build should work much faster than it does.
Please create an issue in the issue tracker and attach generated \*.snapshot files to it or upload them to Please create an issue in the issue tracker and attach generated \*.snapshot files to it or upload them to
[ftp://ftp.intellij.net/.uploads](ftp://ftp.intellij.net/.uploads) and specify links in the issue. [ftp://ftp.intellij.net/.uploads](ftp://ftp.intellij.net/.uploads) and specify links in the issue.
@ -95,24 +95,24 @@ Please also provide details about the memory and other VM settings for the build
The log file is located under the directory The log file is located under the directory
``` ```
<idea-system-directory>/log/build-log <ide-system-directory>/log/build-log
``` ```
There both "build-log.log" and "build-log.xml" files can be found. There both `build-log.log` and `build-log.xml` files can be found.
The build-log.xml is a log4j configuration file, where the log level and desired logging categories can be adjusted. The `build-log.xml` is a log4j configuration file, where the log level and desired logging categories can be adjusted.
This file contains logging from all build sessions, including those from the auto-make. This file contains logging from all build sessions, including those from the auto-make.
### Accessing project model and configuration from External Build ### Accessing project model and configuration from External Build
The project model in External Build process is provided by JPS (JetBrains Project System). The project model in External Build process is provided by JPS (*JetBrains Project System*).
A project is represented by JpsProject, a module by JpsModule and so on. A project is represented by `JpsProject`, a module by `JpsModule` and so on.
If your compiler depends on something that isn't added to the model yet (e.g. some facet settings), If your compiler depends on something that isn't added to the model yet (e.g. some facet settings),
you need to extend the JPS model (use JpsGwtModuleExtension as a reference implementation) and provide implementation of you need to extend the JPS model (use `JpsGwtModuleExtension` as a reference implementation) and provide implementation of
JpsModelSerializerExtension to load the configuration from project files. `JpsModelSerializerExtension` to load the configuration from project files.
#### Implementing builder #### Implementing builder
If your compiler isn't involved into compilation of an existing BuildTarget you need to create a new implementation of BuildTarget and BuildTargetType. Also register an implementation of BuildTargetScopeProvider extension on IDEA side to add required targets to the build scope. If your compiler isn't involved into compilation of an existing `BuildTarget` you need to create a new implementation of `BuildTarget` and `BuildTargetType`. Also register an implementation of `BuildTargetScopeProvider` extension on IDE side to add required targets to the build scope.
The builder implementation should extend either TargetBuilder or ModuleLevelBuilder class and should be created using BuilderService extension. The builder implementation should extend either `TargetBuilder` or `ModuleLevelBuilder` class and should be created using `BuilderService` extension.