hxuanyu/intellij-sdk-code-samples
1
0
Fork 0
mirror of https://github.com/JetBrains/intellij-sdk-code-samples.git synced 2025-07-28 01:07:49 +08:00
Code Issues Packages Projects Releases Wiki Activity

Testing plugins: revise, update, cosmetics

Browse Source
This commit is contained in:
Yann Cébron 2021-09-08 17:42:52 +02:00
parent 6a18ef8382
commit 0e840bc3fe
6 changed files with 51 additions and 42 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
topics/basics/testing_plugins/light_and_heavy_tests.md
View File

@ -1,11 +1,11 @@
[//]: # (title: Light and Heavy Tests)
<!-- Copyright 2000-2020 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
<!-- Copyright 2000-2021 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
Plugin tests run in a real, rather than mocked, IntelliJ Platform environment and use real implementations for most application and project components/services.
Loading and initializing all the project components and services for a project to run tests is a relatively expensive operation, and we want to avoid doing it for each test.
Dependently on the loading and execution time, we make a difference between *heavy* tests and *light* tests available in IntelliJ Platform test framework:
Dependently on the loading and execution time, we make a difference between *heavy* tests and *light* tests available in the IntelliJ Platform test framework:
* *Heavy* tests create a new project for each test.
* *Light* tests reuse a project from the previous test run when possible.
@ -20,17 +20,28 @@ Light and heavy tests use different base classes or fixture classes, as describe
The standard way of writing a light test is to extend the following classes:
<tabs>
<tab title="2019.2 and later">
[`BasePlatformTestCase`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/BasePlatformTestCase.java) for tests that don't have any Java dependencies (previously `LightPlatformCodeInsightFixtureTestCase`).
[`LightJavaCodeInsightFixtureTestCase`](upsource:///java/testFramework/src/com/intellij/testFramework/fixtures/LightJavaCodeInsightFixtureTestCase.java) for tests that require the Java PSI or any related functionality (previously `LightCodeInsightFixtureTestCase`).
</tab>
<tab title="Pre-2019.2">
* [`LightPlatformCodeInsightFixtureTestCase`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/LightPlatformCodeInsightFixtureTestCase.java) for tests that don't have any Java dependencies.
* [`LightCodeInsightFixtureTestCase`](upsource:///java/testFramework/src/com/intellij/testFramework/fixtures/LightCodeInsightFixtureTestCase.java) for tests that require the Java PSI or any related functionality.
> In 2019.2, `LightPlatformCodeInsightFixtureTestCase` has been renamed to [`BasePlatformTestCase`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/BasePlatformTestCase.java) and `LightCodeInsightFixtureTestCase` to [`LightJavaCodeInsightFixtureTestCase`](upsource:///java/testFramework/src/com/intellij/testFramework/fixtures/LightJavaCodeInsightFixtureTestCase.java) respectively.
>
{type="note"}
</tab>
</tabs>
When writing a light test, you can specify the project's requirements that you need to have in your test, such as the module type, the configured SDK, facets, libraries, etc.
You do so by extending the [`LightProjectDescriptor`](upsource:///platform/testFramework/src/com/intellij/testFramework/LightProjectDescriptor.java) class and returning your project descriptor from `getProjectDescriptor()`.
Before executing each test, the project instance will be reused if the test case returns the same project descriptor (usually stored in static final field) as the previous one, or recreated if the descriptor is different (`equals() = false`).
Before executing each test, the project instance will be reused if the test case returns the same project descriptor (usually stored in `static final` field) as the previous one or recreated if the descriptor is different (`equals() = false`).
## Heavy Tests

30
topics/basics/testing_plugins/test_project_and_testdata_directories.md
View File

@ -1,44 +1,44 @@
[//]: # (title: Test Project and Testdata Directories)
<!-- Copyright 2000-2020 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
<!-- Copyright 2000-2021 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
The test fixture creates a *test project* environment.
Unless you customize the project creation, the test project will have one module with one source root called `src`.
Unless you customize the project creation, the test project will have one module with one source root called <path>src</path>.
The test project files exist either in a temporary directory or in an in-memory file system, depending on which implementation of [`TempDirTestFixture`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/TempDirTestFixture.java) is used.
[`LightPlatformCodeInsightFixtureTestCase`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/LightPlatformCodeInsightFixtureTestCase.java) (renamed to `BasePlatformTestCase` in 2019.2) uses an in-memory implementation; if you set up the test environment by calling `IdeaTestFixtureFactory.createCodeInsightFixture()`, you can specify the implementation to use.
[`BasePlatformTestCase`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/BasePlatformTestCase.java) (renamed from `LightPlatformCodeInsightFixtureTestCase` in 2019.2) uses an in-memory implementation; if you set up the test environment by calling `IdeaTestFixtureFactory.createCodeInsightFixture()`, you can specify the implementation to use.
> If your tests use the in-memory implementation, and you abort the execution of your tests, the persisted filesystem caches may get out of sync with the in-memory structures, and you may get spurious errors in your tests.
> If you get an unexpected error after a series of successful runs, **try rerunning the test**, and if that doesn't help, **delete the "system" subdirectory** in your [sandbox directory](ide_development_instance.md#the-development-instance-sandbox-directory).
> If you get an unexpected error after a series of successful runs, **try rerunning the test**, and if that doesn't help, **delete the "system" subdirectory** in your [sandbox directory](ide_development_instance.md#the-development-instance-sandbox-directory).
>
{type="warning"}
In your plugin, you usually store the test data for your tests (such as files on which plugin features will be executed and expected output files) in the `testdata` directory.
In your plugin, you usually store the test data for your tests (such as files on which plugin features will be executed and expected output files) in the <path>testdata</path> directory.
This is just a directory under your plugin's content root, but not under a source root.
Files in `testdata` usually are not valid source code and must not be compiled.
Files in <path>testdata</path> usually are not valid source code and must not be compiled.
To specify the location of `testdata`, you must override the `getTestDataPath()` method.
To specify the location of <path>testdata</path>, you must override the `getTestDataPath()` method.
The default implementation assumes running as part of the IntelliJ Platform source tree and is not appropriate for third-party plugins.
> A very common pattern in IntelliJ Platform tests is to use the test method's name being executed as the base for building the `testdata` file paths.
> This allows us to reuse most of the code between different test methods that test various aspects of the same feature, and this approach is also recommended for third-party plugin tests.
> The name of the test method can be retrieved using `UsefulTestCase.getTestName()`.
> A widespread pattern in IntelliJ Platform tests is to use the test method's name being executed as the base for building the <path>testdata</path> file paths.
> This allows us to reuse most of the code between different test methods that test various aspects of the same feature, and this approach is also recommended for third-party plugin tests.
> The name of the test method can be retrieved using `UsefulTestCase.getTestName()`.
>
{type="note"}
> If your plugin builds on top of Java support, please see [Tests Prerequisites](tests_prerequisites.md) for setting up your test environment to obtain required _Mock JDK_ automatically.
> If your plugin builds on top of Java support, please see [Tests Prerequisites](tests_prerequisites.md) to set up your test environment to obtain the required _Mock JDK_ automatically.
>
{type="note"}
To copy files or directories from your `testdata` directory to the test project directory, you can use the `copyFileToProject()` and `copyDirectoryToProject()` methods in the [`CodeInsightTestFixture`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/CodeInsightTestFixture.java) class.
To copy files or directories from your <path>testdata</path> directory to the test project directory, you can use the `copyFileToProject()` and `copyDirectoryToProject()` methods from [`CodeInsightTestFixture`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/CodeInsightTestFixture.java).
Most operations in plugin tests require a file open in the in-memory editor, in which highlighting, completion, and other operations will be performed.
The in-memory editor instance is returned by `CodeInsightTestFixture.getEditor()`.
To copy a file from the `testdata` directory to the test project directory and immediately open it in the editor, you can use the `CodeInsightTestFixture.configureByFile()` or `configureByFiles()` methods.
To copy a file from the <path>testdata</path> directory to the test project directory and immediately open it in the editor, you can use the `CodeInsightTestFixture.configureByFile()` or `configureByFiles()` methods.
The latter copies multiple files to the test project directory and opens the *first* of them in the in-memory editor.
Alternatively, you can use one of the other methods which take parameters annotated with [`@TestDataFile`](upsource:///platform/testFramework/src/com/intellij/testFramework/TestDataFile.java).
These methods copy the specified files from the `testdata` directory to the test project directory, open the first of the specified files in the in-memory editor, and then perform the requested operation such as highlighting or code completion.
Alternatively, you can use one of the other methods, which take parameters annotated with [`@TestDataFile`](upsource:///platform/testFramework/src/com/intellij/testFramework/TestDataFile.java).
These methods copy the specified files from the <path>testdata</path> directory to the test project directory, open the first of the specified files in the in-memory editor, and then perform the requested operation such as highlighting or code completion.
> The IDE supports smart navigation between test code and related test data file(s); see this [blog post](https://blog.jetbrains.com/platform/2017/10/improvements-in-testing-intellij-platform-plugins/) for more details.
>

12
topics/basics/testing_plugins/testing_highlighting.md
View File

@ -1,13 +1,13 @@
[//]: # (title: Testing Highlighting)
<!-- Copyright 2000-2020 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
<!-- Copyright 2000-2021 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
When writing plugin tests, a common task is testing various kinds of highlighting (inspections, annotators, parser error highlighting, etc.).
The IntelliJ Platform provides a dedicated utility and markup format for this task.
To test the highlighting for the file currently loaded into the in-memory editor, you invoke the `checkHighlighting()` method.
The parameters to the method specify which severities should be taken into account when comparing the results with the expected results: errors are always taken into account, whereas warnings, weak warnings, and infos are optional.
Alternatively, you can use the `testHighlighting()` method, which loads a `testdata` file into the in-memory editor and highlights it as a single operation.
Alternatively, you can use the `testHighlighting()` method, which loads a <path>testdata</path> file into the in-memory editor and highlights it as a single operation.
If you need to test inspections (rather than generic highlighting provided by a highlighting lexer or annotator), you need to enable inspections that you're testing.
This is done by calling `CodeInsightTestFixture.enableInspections()` in the setup method of your test or directly in a test method, before the call to `checkHighlighting()`.
@ -44,9 +44,9 @@ The tag can also have the following optional attributes:
* `descr` expected message associated with the highlighter (if not specified, any text will match; if the message contains a quotation mark, it can be escaped by putting two backslash characters before it)
* `foregroundColor`, `backgroundColor`, `effectColor` expected colors for the highlighting
* `effectType` expected effect type for the highlighting (see [`EffectType`](upsource:///platform/core-api/src/com/intellij/openapi/editor/markup/EffectType.java))
* `fontType` expected font style for the highlighting (0 - normal, 1 - bold, 2 - italic, 3 - bold italic)
* `fontType` expected font style for the highlighting (`0` - normal, `1` - bold, `2` - italic, `3` - bold italic)
> *Nested* tags are **supported**:
> ```<warning>warning_highlight<info>warning_and_info_highlight</info>warning_highlight</warning>```
> *Overlapping* tags (annotations) are currently **not supported** in the test framework (but display correctly in the editor, albeit this is not an officially supported scenario):
> ```<warning>warning_highlight<info>warning-and_info_highlight</warning>info_highlight</info>```
> ```<warning>warning_highlight<info>warning_and_info_highlight</info>warning_highlight</warning>```
> *Overlapping* tags (annotations) are currently **not supported** in the test framework (but display correctly in the editor, albeit this is not an officially supported scenario):
> ```<warning>warning_highlight<info>warning-and_info_highlight</warning>info_highlight</info>```

10
topics/basics/testing_plugins/testing_plugins.md
View File

@ -2,18 +2,18 @@
<!-- Copyright 2000-2021 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
Most of the tests in the IntelliJ Platform codebase are *model level functional tests*.
Most of the tests in the IntelliJ Platform codebase are *model-level functional tests*.
What this means is the following:
* The tests run in a headless environment that uses real production implementations for most components, except for many UI components.
* The tests usually test a feature as a whole, rather than individual functions that comprise its implementation.
* The tests usually test a feature as a whole rather than individual functions that comprise its implementation.
* The tests do not test the Swing UI and work directly with the underlying model instead.
* Most of the tests take a source file or a set of source files as input data, execute a feature, and then compare the output with expected results.
* Most tests take a source file or a set of source files as input data, execute a feature, and compare the output with expected results.
Results can be specified as another set of source files, special markup in the input file, or directly in the test code.
The most significant benefit of this test approach is that tests are very stable and require very little maintenance once written, no matter how much the underlying implementation is refactored or rewritten.
In a product with 15+ years of a lifetime that has gone through a large number of internal refactorings, we find that this benefit dramatically outweighs the downsides of slower test execution and more difficult debugging of failures being compared to more isolated unit tests.
In a product with 15+ years of a lifetime that has gone through many internal refactorings, we find that this benefit dramatically outweighs the downsides of slower test execution and more difficult debugging of failures compared to more isolated unit tests.
### Mocks
@ -24,7 +24,7 @@ We recommend working with real components instead.
### UI Tests
Please see dedicated [intellij-ui-test-robot](https://github.com/JetBrains/intellij-ui-test-robot) library.
Please see the dedicated [intellij-ui-test-robot](https://github.com/JetBrains/intellij-ui-test-robot) library.
It is fully integrated with Gradle-based setup via `runIdeForUiTests` task.
Please do not use _platform/testGuiFramework_; it is reserved for internal use.

10
topics/basics/testing_plugins/tests_and_fixtures.md
View File

@ -1,13 +1,11 @@
[//]: # (title: Tests and Fixtures)
<!-- Copyright 2000-2020 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
<!-- Copyright 2000-2021 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
The IntelliJ Platform testing infrastructure is not tied to any specific test framework.
In fact, the IntelliJ IDEA Team uses JUnit, TestNG, and Cucumber for testing different parts of the project.
However, most of the tests are written using JUnit 3.
The IntelliJ Platform testing infrastructure is not tied to any specific test framework. In fact, the IntelliJ IDEA Team uses [JUnit](https://junit.org), [TestNG](https://testng.org), and [Cucumber](https://cucumber.io/) for testing different parts of the project. However, most of the tests are written using JUnit 3.
When writing your tests, you have the choice between using a standard base class to perform the test set up for you and using a fixture class, which lets you perform the setup manually and does not tie you to a specific test framework.
With the former approach, you can use classes such as [`LightPlatformCodeInsightFixtureTestCase`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/LightPlatformCodeInsightFixtureTestCase.java) (renamed to [`BasePlatformTestCase`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/BasePlatformTestCase.java) in 2019.2).
With the former approach, you can use classes such as [`BasePlatformTestCase`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/BasePlatformTestCase.java) ([`LightPlatformCodeInsightFixtureTestCase`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/LightPlatformCodeInsightFixtureTestCase.java) before 2019.2).
With the latter approach, you use the [`IdeaTestFixtureFactory`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/IdeaTestFixtureFactory.java) class to create instances of fixtures for the test environment, and you need to call the fixture creation and setup methods from the test setup method used by your test framework.
With the latter approach, you use the [`IdeaTestFixtureFactory`](upsource:///platform/testFramework/src/com/intellij/testFramework/fixtures/IdeaTestFixtureFactory.java) class to create instances of fixtures for the test environment. You need to call the fixture creation and setup methods from the test setup method used by your test framework.

4
topics/basics/testing_plugins/writing_tests.md
View File

@ -1,6 +1,6 @@
[//]: # (title: Writing Tests)
<!-- Copyright 2000-2020 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
<!-- Copyright 2000-2021 JetBrains s.r.o. and other contributors. Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE file. -->
In most cases, once you have the necessary files copied to the test project and loaded into the in-memory editor, writing the test itself involves invoking your plugin code and has few dependencies on the test framework.
@ -8,7 +8,7 @@ However, for many common cases, the framework provides helper methods that can m
* `type()` simulates the typing of a character or string into the in-memory editor.
* `performEditorAction()` simulates the execution of an action in the in-memory editor context.
* `complete()` simulates code completion invocation and returns the list of lookup elements displayed in the completion list (or `null` if the completion had no suggestions or one suggestion which was auto-inserted).
* `findUsages()` simulates the invocation of 'Find Usages' and returns the found usages.
* `findUsages()` simulates the invocation of _Find Usages_ and returns the found usages.
* `findSingleIntention()` in combination with `launchAction()` simulate the invocation of an intention action or inspection quick fix with the specified name.
* `renameElementAtCaret()` or `rename()` simulate the execution of a rename refactoring.