Merge branch 'multiple_configurations' into 'master'

Support Multiple configuration

See merge request kosit/validator!41
This commit is contained in:
Andreas Penski 2021-05-21 11:16:20 +00:00
commit 9973304a2a
59 changed files with 1136 additions and 608 deletions

2
.idea/compiler.xml generated
View file

@ -7,8 +7,8 @@
<sourceOutputDir name="target/generated-sources/annotations" />
<sourceTestOutputDir name="target/generated-test-sources/test-annotations" />
<outputRelativeToContentRoot value="true" />
<module name="validationtool" />
<module name="validator" />
<module name="validationtool" />
</profile>
</annotationProcessing>
</component>

View file

@ -5,6 +5,18 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
# 1.5.0 (to be released)
### Added
- Support for multiple configurations and multiple repositories. See [cli documentation](docs/cli.md) for details
- Possibility to use preconfigured Saxon `Processor` instance for validation
### Changed
- [ResolvingConfigurationStrategy.java#getProcessor()](de/kosit/validationtool/api/ResolvingConfigurationStrategy) is removed.
- [INTERNAL] CLI parsing based on pico-cli, commons-cli is removed
## 1.4.2
### Fixed

View file

@ -174,7 +174,6 @@ When using XML related technologies you are supposed to handle certain security
* disables DTD validation completely
* allows loading/resolving only from a configured local content repository (a specific folder)
* tries to prevent known XML security issues (see [OWASP XML_Security_Cheat_Sheet.html](https://cheatsheetseries.owasp.org/cheatsheets/XML_Security_Cheat_Sheet.html))
* only works with OpenJDK based XML stacks
However, you can configure certain aspects related to resolving and security yourself. The validator uses a single interface for accessing or creating the necessary XML API objects like `SchemaFactory`, `Validator`,`URIResolver` or `Processor`: [ResolvingConfigurationStrategy.java](https://github.com/itplr-kosit/validator/tree/master/src/main/java/de/kosit/validationtool/api/ResolvingConfigurationStrategy.java)

View file

@ -24,16 +24,33 @@ The help option displays further CLI options to customize the process:
java -jar validationtool-<version>-standalone.jar --help
```
You can also use multiple scenario configurations and multiple repositories with resources for these. The validator either supports
supplying the parameters in order or using named configuration. Valid usages are
```shell
# multiple scenarios, implicit repository
java -jar validationtool-<version>-standalone.jar -s <scenario-config-file1> -s <scenario-config-file2> [OPTIONS] [FILE]
# multiple scenarios, single defined repository
java -jar validationtool-<version>-standalone.jar -s <scenario-config-file1> -s <scenario-config-file2> -r <path-to-repo> [OPTIONS] [FILE]
# multiple scenarios, multiple repositories ordered
java -jar validationtool-<version>-standalone.jar -s <scenario-config-file1> -r <path-to-repo1> -s <scenario-config-file2> -r <path-to-repo2> [OPTIONS] [FILE]
java -jar validationtool-<version>-standalone.jar -s <scenario-config-file1> -s <scenario-config-file2> -r <path-to-repo1> -r <path-to-repo2> [OPTIONS] [FILE]
# multiple scenarios, multiple repositories (named)
java -jar validationtool-<version>-standalone.jar -s "NAME1=<scenario-config-file1>" -s "NAME2=<scenario-config-file2>" -r "NAME1=<path-to-repo1>" -r "NAME2=<path-to-repo2>" [OPTIONS] [FILE]
```
## Special features
Besides the obvious functionality of validating, the cli provides additional functionality to customize the processing:
|name | option | description |
| - | - | - |
| [Daemon mode](daemon.md) | `-D` | Starts the validator in daemon mode as an HTTP service |
| print mode | `-p` | Print the report to stdout |
| extract html | `-h` | Extracts any html blocks within the report and saves the content to the filesystem. Note: the file name is derived from the node name the html appears in |
| print memory stats | `-m` | Prints some memory usage information. Mainly for debugging purposes on processing huge xml files |
| check assertions | `-c <file>` | Check assertions on the generated reports. This is mainly useful for scenario developers. Ask KoSIT for documentation, if you want to use this feauture |
|name | option | description | | - | - | - | | [Daemon mode](daemon.md) | `-D` | Starts the validator in daemon mode as an HTTP service | |
print mode | `-p` | Print the report to stdout | | extract html | `-h` | Extracts any html blocks within the report and saves the content to
the filesystem. Note: the file name is derived from the node name the html appears in | | print memory stats | `-m` | Prints some memory
usage information. Mainly for debugging purposes on processing huge xml files | | check assertions | `-c <file>` | Check assertions on the
generated reports. This is mainly useful for scenario developers. Ask KoSIT for documentation, if you want to use this feauture |
## Return codes

View file

@ -21,6 +21,6 @@
<modelVersion>4.0.0</modelVersion>
<groupId>de.kosit.validationtool</groupId>
<artifactId>packaged-test-scenarios</artifactId>
<version>1.0.1</version>
<version>1.0.2</version>
<description>POM was created from install:install-file</description>
</project>

View file

@ -19,10 +19,10 @@
<groupId>de.kosit.validationtool</groupId>
<artifactId>packaged-test-scenarios</artifactId>
<versioning>
<release>1.0.1</release>
<release>1.0.2</release>
<versions>
<version>1.0.1</version>
<version>1.0.2</version>
</versions>
<lastUpdated>20200507064929</lastUpdated>
<lastUpdated>20201007064929</lastUpdated>
</versioning>
</metadata>

View file

@ -87,9 +87,9 @@
<version>${version.slf4j}</version>
</dependency>
<dependency>
<groupId>commons-cli</groupId>
<artifactId>commons-cli</artifactId>
<version>1.4</version>
<groupId>info.picocli</groupId>
<artifactId>picocli</artifactId>
<version>4.5.1</version>
<optional>true</optional>
</dependency>
<dependency>
@ -148,7 +148,7 @@
<dependency>
<groupId>de.kosit.validationtool</groupId>
<artifactId>packaged-test-scenarios</artifactId>
<version>1.0.1</version>
<version>1.0.2</version>
<scope>test</scope>
</dependency>

View file

@ -27,12 +27,13 @@ import lombok.extern.slf4j.Slf4j;
import de.kosit.validationtool.impl.ContentRepository;
import de.kosit.validationtool.impl.Scenario;
import de.kosit.validationtool.impl.xml.ProcessorProvider;
/**
* Zentrale Konfigration einer Prüf-Instanz.
*
* @author Andreas Penski
* @deprecated since 1.3.0 use {@link Configuration} instead
* @deprecated since 1.3.0 use {@link Configuration} instead. Will be removed in 2.0
*/
@Getter
@Setter
@ -55,7 +56,7 @@ public class CheckConfiguration implements Configuration {
private Configuration getDelegate() {
if (this.delegate == null) {
this.delegate = Configuration.load(this.scenarioDefinition, this.scenarioRepository).build();
this.delegate = Configuration.load(this.scenarioDefinition, this.scenarioRepository).build(ProcessorProvider.getProcessor());
}
return this.delegate;
}

View file

@ -81,7 +81,7 @@ public interface Configuration {
* Add some additional parameters to the validator configuration. Parameter usage depends on actual implementation
* of {@link Check}
*
* @return
* @return A Map containing the additional Parameters to be added.
*/
Map<String, Object> getAdditionalParameters();

View file

@ -24,7 +24,6 @@ import javax.xml.validation.SchemaFactory;
import javax.xml.validation.Validator;
import net.sf.saxon.lib.UnparsedTextURIResolver;
import net.sf.saxon.s9api.Processor;
/**
* Centralized construction and configuration of XML related infrastructure components. This interface allows to use
@ -50,17 +49,6 @@ public interface ResolvingConfigurationStrategy {
*/
SchemaFactory createSchemaFactory();
/**
* Returns a preconfigured {@link Processor Saxon Processor} for various tasks within the Validator. The validator
* leverages the saxon s9api for internal processing e.g. xml reading and writing. So this is the main object to
* secure for reading, transforming and writing xml files.
*
* Note: you need exactly one instance for all validator related processing.
*
* @return a preconfigured {@link Processor}
*/
Processor getProcessor();
/**
* Creates a specific implementation for resolving referenced objects in XML files. The URIResolver is used for
* dereferencing an absolute URI (after resolution) to return a {@link javax.xml.transform.Source}. It <b>can</b> be

View file

@ -16,25 +16,19 @@
package de.kosit.validationtool.cmd;
import static de.kosit.validationtool.cmd.CommandLineOptions.HELP;
import static de.kosit.validationtool.cmd.CommandLineOptions.createHelpOptions;
import static de.kosit.validationtool.cmd.CommandLineOptions.createOptions;
import static de.kosit.validationtool.cmd.CommandLineOptions.printHelp;
import static de.kosit.validationtool.impl.Printer.writeErr;
import java.util.Arrays;
import org.apache.commons.cli.CommandLine;
import org.apache.commons.cli.CommandLineParser;
import org.apache.commons.cli.DefaultParser;
import org.apache.commons.cli.Options;
import org.apache.commons.cli.ParseException;
import org.apache.commons.lang3.ObjectUtils;
import org.fusesource.jansi.AnsiConsole;
import org.fusesource.jansi.AnsiConsole;
import org.fusesource.jansi.AnsiRenderer.Code;
import de.kosit.validationtool.cmd.report.Line;
import de.kosit.validationtool.impl.Printer;
import picocli.CommandLine;
import picocli.CommandLine.ParseResult;
/**
* Commandline interface of the validator. It parses the commandline args and hands over actual execution to
* {@link Validator}.
@ -59,7 +53,9 @@ public class CommandLineApplication {
AnsiConsole.systemInstall();
final ReturnValue resultStatus = mainProgram(args);
if (!resultStatus.equals(ReturnValue.DAEMON_MODE)) {
sayGoodby(resultStatus);
if (!resultStatus.equals(ReturnValue.HELP_REQUEST) && resultStatus.getCode() >= 0) {
sayGoodby(resultStatus);
}
System.exit(resultStatus.getCode());
} else {
Runtime.getRuntime().addShutdownHook(new Thread(() -> Printer.writeOut("Shutting down daemon ...")));
@ -79,60 +75,37 @@ public class CommandLineApplication {
// for testing purposes. Unless jvm is terminated during tests. See above
static ReturnValue mainProgram(final String[] args) {
final Options options = createOptions();
ReturnValue resultStatus;
final CommandLine commandLine = new CommandLine(new CommandLineOptions());
try {
if (isHelpRequested(args)) {
printHelp(options);
resultStatus = ReturnValue.SUCCESS;
commandLine.setExecutionExceptionHandler(CommandLineApplication::logExecutionException);
commandLine.execute(args);
if (commandLine.isUsageHelpRequested()) {
resultStatus = ReturnValue.HELP_REQUEST;
} else {
final CommandLineParser parser = new DefaultParser();
final CommandLine cmd = parser.parse(options, args);
configureLogging(cmd);
resultStatus = Validator.mainProgram(cmd);
resultStatus = ObjectUtils.defaultIfNull(commandLine.getExecutionResult(), ReturnValue.PARSING_ERROR);
if (resultStatus.getCode() != ReturnValue.PARSING_ERROR.getCode()
&& resultStatus.getCode() != ReturnValue.SUCCESS.getCode()) {
commandLine.usage(System.out);
}
}
} catch (final ParseException e) {
} catch (final Exception e) {
writeErr("Error processing command line arguments: {0}", e.getMessage(), e);
printHelp(options);
resultStatus = ReturnValue.PARSING_ERROR;
}
return resultStatus;
}
private static boolean isHelpRequested(final String[] args) {
final Options helpOptions = createHelpOptions();
try {
final CommandLineParser parser = new DefaultParser();
final CommandLine cmd = parser.parse(helpOptions, args, true);
if (cmd.hasOption(HELP.getOpt()) || args.length == 0) {
return true;
}
} catch (final ParseException e) {
// we can ignore that, we just look for the help parameters
}
return false;
private static int logExecutionException(final Exception ex, final CommandLine cli, final ParseResult parseResult) {
Printer.writeErr(ex, ex.getMessage());
return 1;
}
private static void configureLogging(final CommandLine cmd) throws ParseException {
if (cmd.hasOption(CommandLineOptions.DEBUG_LOG.getOpt())) {
System.setProperty(org.slf4j.impl.SimpleLogger.DEFAULT_LOG_LEVEL_KEY, "DEBUG");
} else if (cmd.hasOption(CommandLineOptions.LOG_LEVEL.getOpt())) {
final String level = Level.resolve(cmd.getOptionValue(CommandLineOptions.LOG_LEVEL.getOpt()));
System.setProperty(org.slf4j.impl.SimpleLogger.DEFAULT_LOG_LEVEL_KEY, level);
} else {
System.setProperty(org.slf4j.impl.SimpleLogger.DEFAULT_LOG_LEVEL_KEY, "OFF");
}
}
private enum Level {
enum Level {
INFO, WARN, DEBUG, TRACE, ERROR, OFF;
static String resolve(final String optionValue) throws ParseException {
return Arrays.stream(values()).filter(e -> e.name().equalsIgnoreCase(optionValue)).map(Enum::name).findFirst()
.orElseThrow(() -> new ParseException("Either specify trace,debug,info,warn,error as log level"));
}
}
}

View file

@ -16,100 +16,176 @@
package de.kosit.validationtool.cmd;
import org.apache.commons.cli.HelpFormatter;
import org.apache.commons.cli.Option;
import java.nio.file.Path;
import java.util.List;
import java.util.concurrent.Callable;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import de.kosit.validationtool.cmd.CommandLineApplication.Level;
import picocli.CommandLine.ArgGroup;
import picocli.CommandLine.Command;
import picocli.CommandLine.Help.Visibility;
import picocli.CommandLine.Option;
import picocli.CommandLine.Parameters;
/**
* Commandline Interface definition.
*
* @author Andreas Penski
*/
public class CommandLineOptions {
@Command(description = "Structural and semantic validation of xml files", name = "KoSIT Validator", mixinStandardHelpOptions = false,
separator = " ")
@Getter
public class CommandLineOptions implements Callable<ReturnValue> {
static final Option HELP = Option.builder("?").longOpt("help").argName("Help").desc("Displays this help").build();
/**
* @author Andreas Penski
*/
@Getter
@NoArgsConstructor
static class DaemonOptions {
static final Option SCENARIOS = Option.builder("s").required().longOpt("scenarios").hasArg().desc("Location of scenarios.xml e.g.")
.build();
@Option(names = { "-D", "--daemon" }, description = "Starts a daemon listing for validation requests", defaultValue = "false",
required = true)
private boolean daemonMode;
static final Option REPOSITORY = Option.builder("r").longOpt("repository").hasArg().desc("Directory containing scenario content")
.build();
@Option(names = { "-H", "--host" }, description = "The hostname / IP address to bind the daemon.", defaultValue = "localhost",
showDefaultValue = Visibility.ALWAYS)
private String host;
static final Option PRINT = Option.builder("p").longOpt("print").desc("Prints the check result to stdout").build();
@Option(names = { "-P", "--port" }, description = "The port to bind the daemon.", defaultValue = "8080",
showDefaultValue = Visibility.ALWAYS)
private int port;
static final Option OUTPUT = Option.builder("o").longOpt("output-directory")
.desc("Defines the out directory for results. Defaults to cwd").hasArg().build();
@Option(names = { "-T", "--threads" },
description = "Number of threads processing validation requests. Default depends on processor count", defaultValue = "-1",
showDefaultValue = Visibility.NEVER)
private int workerCount;
static final Option EXTRACT_HTML = Option.builder("h").longOpt("html")
.desc("Extract and save any html content within result as a separate file ").build();
static final Option DEBUG = Option.builder("d").longOpt("debug").desc("Prints some more debug information").build();
static final Option SERIALIZE_REPORT_INPUT = Option.builder("c").longOpt("serialize-report-input")
.desc("Serializes the report input to the cwd").build();
static final Option CHECK_ASSERTIONS = Option.builder("c").longOpt("check-assertions").hasArg()
.desc("Check the result using defined assertions").argName("assertions-file").build();
static final Option SERVER = Option.builder("D").longOpt("daemon").desc("Starts a daemon listing for validation requests").build();
static final Option HOST = Option.builder("H").longOpt("host").hasArg()
.desc("The hostname / IP address to bind the daemon. Default is localhost").build();
static final Option PORT = Option.builder("P").longOpt("port").hasArg().desc("The port to bind the daemon. Default is 8080").build();
static final Option WORKER_COUNT = Option.builder("T").longOpt("threads").hasArg()
.desc("Number of threads processing validation requests").build();
static final Option DISABLE_GUI = Option.builder("G").longOpt("disable-gui").desc("Disables the GUI of the daemon mode").build();
static final Option REPORT_POSTFIX = Option.builder(null).longOpt("report-postfix").hasArg()
.desc("Postfix of the generated report name").build();
static final Option REPORT_PREFIX = Option.builder(null).longOpt("report-prefix").hasArg().desc("Prefix of the generated report name")
.build();
static final Option DEBUG_LOG = Option.builder("X").longOpt("debug-logging").desc("Enables full debug log. Alias for -l debug").build();
static final Option LOG_LEVEL = Option.builder("l").longOpt("log-level").hasArg()
.desc("Enables a certain log level for debugging " + "purposes").build();
static final Option PRINT_MEM_STATS = Option.builder("m").longOpt("memory-stats").desc("Prints some memory stats").build();
private CommandLineOptions() {
// hide
@Option(names = { "-G", "--disable-gui" }, description = "Disables the GUI of the daemon mode")
private boolean disableGUI;
}
static org.apache.commons.cli.Options createOptions() {
final org.apache.commons.cli.Options options = new org.apache.commons.cli.Options();
options.addOption(HELP);
options.addOption(SERVER);
options.addOption(HOST);
options.addOption(PORT);
options.addOption(SCENARIOS);
options.addOption(REPOSITORY);
options.addOption(PRINT);
options.addOption(OUTPUT);
options.addOption(EXTRACT_HTML);
options.addOption(DEBUG);
options.addOption(CHECK_ASSERTIONS);
options.addOption(PRINT_MEM_STATS);
options.addOption(WORKER_COUNT);
options.addOption(DISABLE_GUI);
options.addOption(REPORT_POSTFIX);
options.addOption(REPORT_PREFIX);
options.addOption(LOG_LEVEL);
options.addOption(DEBUG_LOG);
return options;
/**
* @author Andreas Penski
*/
@Getter
@NoArgsConstructor
static class CliOptions {
@Option(names = { "-o", "--output-directory" }, description = "Defines the out directory for results.", defaultValue = ".",
required = true)
private Path outputPath;
@Option(names = { "-h", "--html", "--extract-html" },
description = "Extract and save any html content within result as a separate file")
private boolean extractHtml;
@Option(names = { "--serialize-report-input" }, description = "Serializes the report input to the cwd", defaultValue = "false")
private boolean serializeInput;
@Option(names = { "-c", "--check-assertions" }, paramLabel = "assertions-file",
description = "Check the result using defined assertions")
private Path assertions;
@Option(names = { "--report-postfix" }, description = "Postfix of the generated report name")
private String reportPostfix;
@Option(names = { "--report-prefix" }, description = "Prefix of the generated report name")
private String reportPrefix;
@Option(names = { "-m", "--memory-stats" }, description = "Prints some memory stats")
private boolean printMemoryStats;
@Option(names = { "-p", "--print" }, description = "Prints the check result to stdout")
private boolean printReport;
@Parameters(arity = "1..*", description = "Files to validate")
private List<Path> files;
}
static void printHelp(final org.apache.commons.cli.Options options) {
// automatically generate the help statement
final HelpFormatter formatter = new HelpFormatter();
formatter.printHelp("check-tool -s <scenario-config-file> [OPTIONS] [FILE]... ", options, false);
/**
* Definition of logical name and a path for a configuration artifact.
*
* @author Andreas Penski
*/
@Getter
@Setter
public abstract static class Definition {
String name;
Path path;
}
static org.apache.commons.cli.Options createHelpOptions() {
final org.apache.commons.cli.Options options = new org.apache.commons.cli.Options();
options.addOption(HELP);
return options;
/**
* Definition of logical name and a path for a repository.
*
* @author Andreas Penski
*/
public static class RepositoryDefinition extends Definition {
// just for type safety
}
/**
* Definition of logical name and a path for a scenario configuration file.
*
* @author Andreas Penski
*/
public static class ScenarioDefinition extends Definition {
// just for type safety
}
@ArgGroup(exclusive = false, heading = "Daemon options\n")
private DaemonOptions daemonOptions;
@ArgGroup(exclusive = false, heading = "CLI usage options\n")
private CliOptions cliOptions;
@Option(names = { "-d", "--debug" }, description = "Prints some more debug information")
private boolean debugOutput;
@Option(names = { "-?", "--help" }, usageHelp = true, description = "display this help message")
boolean usageHelpRequested;
@Option(names = { "-X", "--debug-logging" }, description = "Enables full debug log. Alias for -l debug")
private boolean debugLog;
@Option(names = { "-l", "--log-level" }, description = "Enables a certain log level for debugging purposes", defaultValue = "OFF")
private Level logLevel;
@Option(names = { "-r", "--repository" }, paramLabel = "repository-path", description = "Directory containing scenario content",
converter = TypeConverter.RepositoryConverter.class)
private List<RepositoryDefinition> repositories;
@Option(names = { "-s", "--scenarios" }, description = "Location of scenarios.xml", paramLabel = "scenario.xml", required = true,
converter = TypeConverter.ScenarioConverter.class)
private List<ScenarioDefinition> scenarios;
@Override
public ReturnValue call() throws Exception {
configureLogging(this);
return Validator.mainProgram(this);
}
private static void configureLogging(final CommandLineOptions cmd) {
if (cmd.isDebugLog()) {
System.setProperty(org.slf4j.impl.SimpleLogger.DEFAULT_LOG_LEVEL_KEY, "DEBUG");
} else {
System.setProperty(org.slf4j.impl.SimpleLogger.DEFAULT_LOG_LEVEL_KEY, cmd.getLogLevel().name());
}
}
public boolean isDaemonModeEnabled() {
return getDaemonOptions() != null;
}
public boolean isCliModeEnabled() {
return getCliOptions() != null;
}
}

View file

@ -37,6 +37,8 @@ import de.kosit.validationtool.cmd.report.Line;
import de.kosit.validationtool.impl.DefaultCheck;
import de.kosit.validationtool.impl.tasks.CheckAction;
import net.sf.saxon.s9api.Processor;
/**
* Simple Erweiterung der Klasse {@link DefaultCheck} um das Ergebnis der Assertion-Prüfung auszuwerten und auszugeben.
* Diese Klasse stellt keine fachliche Erweiterung des eigentlichen Prüfvorganges dar!
@ -55,8 +57,8 @@ class InternalCheck extends DefaultCheck {
*
* @param configuration die Konfiguration
*/
InternalCheck(final Configuration configuration) {
super(configuration);
InternalCheck(final Processor processor, final Configuration... configuration) {
super(processor, configuration);
}
/**

View file

@ -30,6 +30,8 @@ public class ReturnValue {
public static final ReturnValue SUCCESS = new ReturnValue(0);
public static final ReturnValue HELP_REQUEST = new ReturnValue(0);
public static final ReturnValue CONFIGURATION_ERROR = new ReturnValue(-2);
public static final ReturnValue DAEMON_MODE = new ReturnValue(-100);
@ -41,4 +43,5 @@ public class ReturnValue {
public static ReturnValue createFailed(final int count) {
return new ReturnValue(count);
}
}

View file

@ -0,0 +1,99 @@
/*
* Copyright 2017-2021 Koordinierungsstelle für IT-Standards (KoSIT)
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package de.kosit.validationtool.cmd;
import static org.apache.commons.lang3.StringUtils.isNotBlank;
import java.nio.file.Paths;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import de.kosit.validationtool.cmd.CommandLineOptions.Definition;
import de.kosit.validationtool.cmd.CommandLineOptions.RepositoryDefinition;
import de.kosit.validationtool.cmd.CommandLineOptions.ScenarioDefinition;
import de.kosit.validationtool.impl.ScenarioRepository;
import picocli.CommandLine.ITypeConverter;
/**
* Custom type converters for dealing with command line input.
*
* @author Andreas Penski
*/
class TypeConverter {
/**
* Type converter for a repository definition specification e.g. '-r somelocation.xml OR -r myid=somelocation.xml'
*
* @author Andreas Penski
*/
public static class RepositoryConverter implements ITypeConverter<RepositoryDefinition> {
@Override
public RepositoryDefinition convert(final String value) throws Exception {
return TypeConverter.convert(RepositoryDefinition.class, value);
}
}
/**
* Type converter for a scenario definition specification e.g. '-s somelocation.xml OR -s myid=somelocation.xml'
*
* @author Andreas Penski
*/
public static class ScenarioConverter implements ITypeConverter<ScenarioDefinition> {
@Override
public ScenarioDefinition convert(final String value) throws Exception {
return TypeConverter.convert(ScenarioDefinition.class, value);
}
}
final static Map<Class<?>, AtomicInteger> counter = new HashMap<>();
private static String getDefaultName(final Class<?> type) {
final AtomicInteger current = counter.computeIfAbsent(type, a -> new AtomicInteger(1));
return ScenarioRepository.DEFAULT + "_" + current.getAndIncrement();
}
private static <T extends Definition> T convert(final Class<T> type, final String value) {
T def = null;
if (isNotBlank(value)) {
final String[] splitted = value.split("=");
if (splitted.length == 1) {
def = createNewInstance(type);
def.setName(getDefaultName(type));
def.setPath(Paths.get(splitted[0].trim()));
} else if (splitted.length == 2) {
def = createNewInstance(type);
def.setName(splitted[0].trim());
def.setPath(Paths.get(splitted[1].trim()));
} else {
throw new IllegalArgumentException("Not a valid repository specification " + value);
}
}
return def;
}
private static <T extends Definition> T createNewInstance(final Class<T> type) {
try {
return type.getConstructor().newInstance();
} catch (final ReflectiveOperationException e) {
throw new IllegalStateException("Error creating instance of type " + type);
}
}
}

View file

@ -16,25 +16,9 @@
package de.kosit.validationtool.cmd;
import static de.kosit.validationtool.cmd.CommandLineOptions.CHECK_ASSERTIONS;
import static de.kosit.validationtool.cmd.CommandLineOptions.DEBUG;
import static de.kosit.validationtool.cmd.CommandLineOptions.DISABLE_GUI;
import static de.kosit.validationtool.cmd.CommandLineOptions.EXTRACT_HTML;
import static de.kosit.validationtool.cmd.CommandLineOptions.HELP;
import static de.kosit.validationtool.cmd.CommandLineOptions.HOST;
import static de.kosit.validationtool.cmd.CommandLineOptions.OUTPUT;
import static de.kosit.validationtool.cmd.CommandLineOptions.PORT;
import static de.kosit.validationtool.cmd.CommandLineOptions.PRINT;
import static de.kosit.validationtool.cmd.CommandLineOptions.PRINT_MEM_STATS;
import static de.kosit.validationtool.cmd.CommandLineOptions.REPORT_POSTFIX;
import static de.kosit.validationtool.cmd.CommandLineOptions.REPORT_PREFIX;
import static de.kosit.validationtool.cmd.CommandLineOptions.REPOSITORY;
import static de.kosit.validationtool.cmd.CommandLineOptions.SCENARIOS;
import static de.kosit.validationtool.cmd.CommandLineOptions.SERIALIZE_REPORT_INPUT;
import static de.kosit.validationtool.cmd.CommandLineOptions.SERVER;
import static de.kosit.validationtool.cmd.CommandLineOptions.WORKER_COUNT;
import static de.kosit.validationtool.cmd.CommandLineOptions.createOptions;
import static de.kosit.validationtool.cmd.CommandLineOptions.printHelp;
import static org.apache.commons.lang3.ObjectUtils.defaultIfNull;
import static org.apache.commons.lang3.StringUtils.EMPTY;
import static org.apache.commons.lang3.StringUtils.isNotEmpty;
import java.io.IOException;
import java.net.URI;
@ -42,18 +26,15 @@ import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.apache.commons.cli.CommandLine;
import org.apache.commons.cli.Option;
import org.apache.commons.lang3.ArrayUtils;
import org.apache.commons.lang3.StringUtils;
import org.fusesource.jansi.AnsiRenderer.Code;
import lombok.extern.slf4j.Slf4j;
@ -62,18 +43,23 @@ import de.kosit.validationtool.api.Configuration;
import de.kosit.validationtool.api.Input;
import de.kosit.validationtool.api.InputFactory;
import de.kosit.validationtool.api.Result;
import de.kosit.validationtool.cmd.CommandLineOptions.CliOptions;
import de.kosit.validationtool.cmd.CommandLineOptions.Definition;
import de.kosit.validationtool.cmd.CommandLineOptions.RepositoryDefinition;
import de.kosit.validationtool.cmd.CommandLineOptions.ScenarioDefinition;
import de.kosit.validationtool.cmd.assertions.Assertions;
import de.kosit.validationtool.cmd.report.Line;
import de.kosit.validationtool.config.ConfigurationLoader;
import de.kosit.validationtool.daemon.Daemon;
import de.kosit.validationtool.impl.ConversionService;
import de.kosit.validationtool.impl.EngineInformation;
import de.kosit.validationtool.impl.Printer;
import de.kosit.validationtool.impl.ScenarioRepository;
import de.kosit.validationtool.impl.xml.ProcessorProvider;
import net.sf.saxon.s9api.Processor;
/**
* Actual evaluation and processing of commandline argumtens.
* Actual evaluation and processing of CommandLineOptions argumtens.
*
* @author Andreas Penski
*/
@ -90,23 +76,23 @@ public class Validator {
*
* @param cmd parsed commandline.
*/
static ReturnValue mainProgram(final CommandLine cmd) {
static ReturnValue mainProgram(final CommandLineOptions cmd) {
greeting();
final org.apache.commons.cli.Options options = createOptions();
final ReturnValue returnValue;
try {
if (cmd.hasOption(SERVER.getOpt())) {
if (cmd.isDaemonModeEnabled()) {
startDaemonMode(cmd);
returnValue = ReturnValue.DAEMON_MODE;
} else if (cmd.hasOption(HELP.getOpt()) || (cmd.getArgList().isEmpty() && !isPiped())) {
printHelp(options);
returnValue = ReturnValue.PARSING_ERROR;
} else {
} else if (cmd.isCliModeEnabled() || isPiped()) {
returnValue = processActions(cmd);
} else {
Printer.writeErr("No test target found");
returnValue = ReturnValue.CONFIGURATION_ERROR;
}
} catch (final Exception e) {
e.printStackTrace();
Printer.writeErr(e.getMessage());
if (cmd.hasOption(DEBUG.getOpt())) {
if (cmd.isDebugOutput()) {
log.error(e.getMessage(), e);
} else {
log.error(e.getMessage());
@ -120,83 +106,55 @@ public class Validator {
Printer.writeOut("{0} version {1}", EngineInformation.getName(), EngineInformation.getVersion());
}
private static int determinePort(final CommandLine cmd) {
int port = 8080;
if (checkOptionWithValue(PORT, cmd)) {
port = Integer.parseInt(cmd.getOptionValue(PORT.getOpt()));
}
return port;
}
private static int determineThreads(final CommandLine cmd) {
private static int determineThreads(final CommandLineOptions.DaemonOptions cmd) {
int threads = Runtime.getRuntime().availableProcessors();
if (checkOptionWithValue(WORKER_COUNT, cmd)) {
threads = Integer.parseInt(cmd.getOptionValue(WORKER_COUNT.getOpt()));
if (cmd.getWorkerCount() > 0) {
threads = cmd.getWorkerCount();
}
return threads;
}
private static String determineHost(final CommandLine cmd) {
String host = "localhost";
if (checkOptionWithValue(HOST, cmd)) {
host = cmd.getOptionValue(HOST.getOpt());
private static void startDaemonMode(final CommandLineOptions cmd) {
if (cmd.isCliModeEnabled()) {
Printer.writeErr("Mixed mode configuration detected. Use either daemon mode or cli mode commandline options. They are mutual "
+ "exclusive. Will ignore cli mode options");
}
return host;
}
private static void startDaemonMode(final CommandLine cmd) {
final Option[] unavailable = new Option[] { PRINT, CHECK_ASSERTIONS, DEBUG, OUTPUT, EXTRACT_HTML, REPORT_POSTFIX, REPORT_PREFIX };
warnUnusedOptions(cmd, unavailable, true);
final ConfigurationLoader config = getConfiguration(cmd);
final Daemon validDaemon = new Daemon(determineHost(cmd), determinePort(cmd), determineThreads(cmd));
if (cmd.hasOption(DISABLE_GUI.getOpt())) {
validDaemon.setGuiEnabled(false);
}
final Configuration configuration = config.build();
printScenarios(configuration);
final List<Configuration> configuration = getConfiguration(cmd);
final CommandLineOptions.DaemonOptions daemonOptions = cmd.getDaemonOptions();
final Daemon validDaemon = new Daemon(daemonOptions.getHost(), daemonOptions.getPort(), determineThreads(daemonOptions));
validDaemon.setGuiEnabled(!daemonOptions.isDisableGUI());
Printer.writeOut("\nStarting daemon mode ...");
validDaemon.startServer(configuration);
validDaemon.startServer(ProcessorProvider.getProcessor(), configuration.toArray(new Configuration[configuration.size()]));
}
private static void warnUnusedOptions(final CommandLine cmd, final Option[] unavailable, final boolean daemon) {
Arrays.stream(cmd.getOptions()).filter(o -> ArrayUtils.contains(unavailable, o))
.map(o -> "The option " + o.getLongOpt() + " is not available in daemon mode").forEach(log::error);
if (daemon && !cmd.getArgList().isEmpty()) {
log.info("Ignoring test targets in daemon mode");
}
}
private static ReturnValue processActions(final CommandLine cmd) throws IOException {
private static ReturnValue processActions(final CommandLineOptions cmd) throws IOException {
long start = System.currentTimeMillis();
final Option[] unavailable = new Option[] { HOST, PORT, WORKER_COUNT, DISABLE_GUI };
warnUnusedOptions(cmd, unavailable, false);
final Configuration config = getConfiguration(cmd).build();
printScenarios(config);
final InternalCheck check = new InternalCheck(config);
final Path outputDirectory = determineOutputDirectory(cmd);
final Processor processor = config.getContentRepository().getProcessor();
if (cmd.hasOption(EXTRACT_HTML.getOpt())) {
final Processor processor = ProcessorProvider.getProcessor();
final List<Configuration> config = getConfiguration(cmd);
final InternalCheck check = new InternalCheck(processor, config.toArray(new Configuration[0]));
final CommandLineOptions.CliOptions cliOptions = defaultIfNull(cmd.getCliOptions(), new CliOptions());
final Path outputDirectory = determineOutputDirectory(cliOptions);
if (cliOptions.isExtractHtml()) {
check.getCheckSteps().add(new ExtractHtmlContentAction(processor, outputDirectory));
}
check.getCheckSteps().add(new SerializeReportAction(outputDirectory, processor, determineNamingStrategy(cmd)));
if (cmd.hasOption(SERIALIZE_REPORT_INPUT.getOpt())) {
check.getCheckSteps().add(new SerializeReportAction(outputDirectory, processor, determineNamingStrategy(cliOptions)));
if (cliOptions.isSerializeInput()) {
check.getCheckSteps().add(new SerializeReportInputAction(outputDirectory, check.getConversionService()));
}
if (cmd.hasOption(PRINT.getOpt())) {
if (cliOptions.isPrintReport()) {
check.getCheckSteps().add(new PrintReportAction(processor));
}
if (cmd.hasOption(CHECK_ASSERTIONS.getOpt())) {
final Assertions assertions = loadAssertions(cmd.getOptionValue(CHECK_ASSERTIONS.getOpt()));
if (cliOptions.getAssertions() != null) {
final Assertions assertions = loadAssertions(cliOptions.getAssertions());
check.getCheckSteps().add(new CheckAssertionAction(assertions, processor));
}
if (cmd.hasOption(PRINT_MEM_STATS.getOpt())) {
if (cliOptions.isPrintMemoryStats()) {
check.getCheckSteps().add(new PrintMemoryStats());
}
log.info("Setup completed in {}ms\n", System.currentTimeMillis() - start);
final Collection<Input> targets = determineTestTargets(cmd);
final Collection<Input> targets = determineTestTargets(cliOptions);
start = System.currentTimeMillis();
final Map<String, Result> results = new HashMap<>();
Printer.writeOut("\nProcessing of {0} objects started", targets.size());
@ -216,42 +174,79 @@ public class Validator {
return check.isSuccessful(results) ? ReturnValue.SUCCESS : ReturnValue.createFailed(check.getNotAcceptableCount(results));
}
private static ConfigurationLoader getConfiguration(final CommandLine cmd) {
final URI scenarioLocation = determineDefinition(cmd);
final URI repositoryLocation = determineRepository(cmd);
reportConfiguration(scenarioLocation, repositoryLocation);
return Configuration.load(scenarioLocation, repositoryLocation);
/**
* @param cmd the Command Line Options
*
* @return a list of configurations of the scenarios and repositories passed in cmd
*/
private static List<Configuration> getConfiguration(final CommandLineOptions cmd) {
final List<ScenarioDefinition> scenarios = defaultIfNull(cmd.getScenarios(), Collections.emptyList());
final Map<String, Path> mappedScenarios = scenarios.stream()
.collect(Collectors.toMap(ScenarioDefinition::getName, ScenarioDefinition::getPath));
final List<RepositoryDefinition> repos = defaultIfNull(cmd.getRepositories(), Collections.emptyList());
final Map<String, Path> mappedRepos = repos.stream().collect(Collectors.toMap(Definition::getName, Definition::getPath));
checkUnused(mappedScenarios, mappedRepos);
return mappedScenarios.entrySet().stream().map(e -> {
assertFileExistance(e.getValue(), "scenario");
final URI scenarioLocation = e.getValue().toUri();
final URI repositoryLocation = findRepository(e.getKey(), mappedRepos);
reportLoading(scenarioLocation, repositoryLocation);
final Configuration configuration = Configuration.load(scenarioLocation, repositoryLocation)
.build(ProcessorProvider.getProcessor());
reportConfiguration(configuration);
return configuration;
}).collect(Collectors.toList());
}
private static void reportConfiguration(final URI scenarioLocation, final URI repositoryLocation) {
private static void checkUnused(final Map<String, Path> scenarios, final Map<String, Path> repositories) {
final List<Entry<String, Path>> unused = repositories.entrySet().stream().filter(e -> scenarios.get(e.getKey()) == null)
.collect(Collectors.toList());
unused.removeIf(e -> e.getKey().equals(ScenarioRepository.DEFAULT_ID));
unused.forEach(e -> Printer.writeErr("Warning: repository definition \"{0}\" is not used", e.getKey()));
}
private static URI findRepository(final String key, final Map<String, Path> repositories) {
final Path path = repositories.getOrDefault(key, repositories.get(ScenarioRepository.DEFAULT_ID));
if (path == null) {
throw new IllegalArgumentException(String.format("No repository location for scenario definition '%s' specified", key));
}
return determineRepository(path);
}
private static void reportLoading(final URI scenarioLocation, final URI repositoryLocation) {
Printer.writeOut("Loading scenarios from {0}", scenarioLocation);
Printer.writeOut("Using repository {0}", repositoryLocation);
Printer.writeOut(EMPTY);
}
private static void printScenarios(final Configuration configuration) {
private static void reportConfiguration(final Configuration configuration) {
Printer.writeOut("Loaded \"{0}\" by {1} from {2} ", configuration.getName(), configuration.getAuthor(), configuration.getDate());
Printer.writeOut("\nThe following scenarios are available:");
Printer.writeOut("The following scenarios are available:");
configuration.getScenarios().forEach(e -> {
final Line line = new Line(Code.GREEN);
line.add(" * " + e.getName());
Printer.writeOut(line.render(false, false));
});
Printer.writeOut(EMPTY);
}
private static NamingStrategy determineNamingStrategy(final CommandLine cmd) {
private static NamingStrategy determineNamingStrategy(final CommandLineOptions.CliOptions cmd) {
final DefaultNamingStrategy namingStrategy = new DefaultNamingStrategy();
if (cmd.hasOption(REPORT_PREFIX.getLongOpt())) {
namingStrategy.setPrefix(cmd.getOptionValue(REPORT_PREFIX.getLongOpt()));
if (isNotEmpty(cmd.getReportPrefix())) {
namingStrategy.setPrefix(cmd.getReportPrefix());
}
if (cmd.hasOption(REPORT_POSTFIX.getLongOpt())) {
namingStrategy.setPostfix(cmd.getOptionValue(REPORT_POSTFIX.getLongOpt()));
if (isNotEmpty(cmd.getReportPostfix())) {
namingStrategy.setPostfix(cmd.getReportPostfix());
}
return namingStrategy;
}
private static Assertions loadAssertions(final String optionValue) {
final Path p = Paths.get(optionValue);
private static Assertions loadAssertions(final Path p) {
Assertions a = null;
if (Files.exists(p)) {
final ConversionService c = new ConversionService();
@ -261,24 +256,23 @@ public class Validator {
return a;
}
private static Path determineOutputDirectory(final CommandLine cmd) {
final String value = cmd.getOptionValue(OUTPUT.getOpt());
final Path fir;
if (StringUtils.isNotBlank(value)) {
fir = Paths.get(value);
if ((!Files.exists(fir) && !fir.toFile().mkdirs()) || !Files.isDirectory(fir)) {
throw new IllegalStateException(String.format("Invalid target directory %s specified", value));
private static Path determineOutputDirectory(final CommandLineOptions.CliOptions cmd) {
final Path dir;
if (cmd.getOutputPath() != null) {
dir = cmd.getOutputPath();
if ((!Files.exists(dir) && !dir.toFile().mkdirs()) || !Files.isDirectory(dir)) {
throw new IllegalStateException(String.format("Invalid target directory %s specified", dir.toString()));
}
} else {
fir = Paths.get(""/* cwd */);
dir = Paths.get(""/* cwd */);
}
return fir;
return dir;
}
private static Collection<Input> determineTestTargets(final CommandLine cmd) throws IOException {
private static Collection<Input> determineTestTargets(final CommandLineOptions.CliOptions cmd) throws IOException {
final Collection<Input> targets = new ArrayList<>();
if (!cmd.getArgList().isEmpty()) {
cmd.getArgList().forEach(e -> targets.addAll(determineTestTarget(e)));
if (cmd.getFiles() != null && !cmd.getFiles().isEmpty()) {
cmd.getFiles().forEach(e -> targets.addAll(determineTestTarget(e)));
}
if (isPiped()) {
targets.add(readFromPipe());
@ -289,22 +283,23 @@ public class Validator {
return targets;
}
@SuppressWarnings("java:S4829") // sanitation is delegated to xml stack
private static boolean isPiped() throws IOException {
return System.in.available() > 0;
}
@SuppressWarnings("java:S4829") // sanitation is delegated to xml stack
private static Input readFromPipe() {
return InputFactory.read(System.in, "stdin");
}
private static Collection<Input> determineTestTarget(final String s) {
final Path d = Paths.get(s);
private static Collection<Input> determineTestTarget(final Path d) {
if (Files.isDirectory(d)) {
return listDirectoryTargets(d);
} else if (Files.exists(d)) {
return Collections.singleton(InputFactory.read(d));
}
log.warn("The specified test target {} does not exist. Will be ignored", s);
log.warn("The specified test target {} does not exist. Will be ignored", d);
return Collections.emptyList();
}
@ -319,44 +314,21 @@ public class Validator {
}
private static URI determineRepository(final CommandLine cmd) {
if (checkOptionWithValue(REPOSITORY, cmd)) {
final Path d = Paths.get(cmd.getOptionValue(REPOSITORY.getOpt()));
if (Files.isDirectory(d)) {
return d.toUri();
} else {
throw new IllegalArgumentException(
String.format("Not a valid path for repository definition specified: '%s'", d.toAbsolutePath()));
}
}
return null;
}
private static URI determineDefinition(final CommandLine cmd) {
checkOptionWithValue(SCENARIOS, cmd);
final Path f = Paths.get(cmd.getOptionValue(SCENARIOS.getOpt()));
if (Files.isRegularFile(f)) {
return f.toAbsolutePath().toUri();
private static URI determineRepository(final Path d) {
if (Files.isDirectory(d)) {
return d.toUri();
} else {
throw new IllegalArgumentException(
String.format("Not a valid path for scenario definition specified: '%s'", f.toAbsolutePath()));
String.format("Not a valid path for repository definition specified: '%s'", d.toAbsolutePath()));
}
}
private static boolean checkOptionWithValue(final Option option, final CommandLine cmd) {
final String opt = option.getOpt();
if (cmd.hasOption(opt)) {
final String value = cmd.getOptionValue(opt);
if (StringUtils.isNoneBlank(value)) {
return true;
} else {
throw new IllegalArgumentException(String.format("Option value required for Option '%s'", option.getLongOpt()));
}
} else if (option.isRequired()) {
throw new IllegalArgumentException(String.format("Option '%s' required ", option.getLongOpt()));
private static void assertFileExistance(final Path f, final String type) {
if (!Files.isRegularFile(f)) {
throw new IllegalArgumentException(
String.format("Not a valid path for %s definition specified: '%s'", type, f.toAbsolutePath()));
}
return false;
}
}

View file

@ -68,8 +68,6 @@ public class ConfigurationBuilder {
private ResolvingMode resolvingMode = ResolvingMode.STRICT_RELATIVE;
private Processor processor;
private String author = "API";
private String date = LocalDate.now().toString();
@ -269,12 +267,9 @@ public class ConfigurationBuilder {
* @return a valid configuration
* @throws IllegalStateException when the configuration is not valid/complete
*/
public Configuration build() {
public Configuration build(final Processor processor) {
final ResolvingConfigurationStrategy resolving = getResolvingConfigurationStrategy();
if (this.processor == null) {
this.processor = resolving.getProcessor();
}
final ContentRepository contentRepository = new ContentRepository(resolving, this.repository);
final ContentRepository contentRepository = new ContentRepository(processor, resolving, this.repository);
final List<Scenario> list = initializeScenarios(contentRepository);
final Scenario fallbackScenario = initializeFallback(contentRepository);

View file

@ -41,6 +41,7 @@ import de.kosit.validationtool.impl.ContentRepository;
import de.kosit.validationtool.impl.ConversionService;
import de.kosit.validationtool.impl.ResolvingMode;
import de.kosit.validationtool.impl.Scenario;
import de.kosit.validationtool.impl.SchemaProvider;
import de.kosit.validationtool.impl.model.Result;
import de.kosit.validationtool.impl.tasks.DocumentParseAction;
import de.kosit.validationtool.impl.xml.RelativeUriResolver;
@ -131,12 +132,11 @@ public class ConfigurationLoader {
}
public Configuration build() {
public Configuration build(final Processor processor) {
final ResolvingConfigurationStrategy resolving = getResolvingConfigurationStrategy();
final Processor processor = resolving.getProcessor();
final ContentRepository contentRepository = new ContentRepository(resolving, getScenarioRepository());
final ContentRepository contentRepository = new ContentRepository(processor, resolving, getScenarioRepository());
final Scenarios def = loadScenarios(contentRepository.getScenarioSchema(), processor);
final Scenarios def = loadScenarios(SchemaProvider.getScenarioSchema(), processor);
final List<Scenario> scenarios = initializeScenarios(def, contentRepository);
final Scenario fallbackScenario = createFallback(def, contentRepository);
final DefaultConfiguration configuration = new DefaultConfiguration(scenarios, fallbackScenario);
@ -164,10 +164,10 @@ public class ConfigurationLoader {
}
private Scenarios loadScenarios(final Schema scenarioSchema, final Processor processor) {
final ConversionService conversionService = new ConversionService();
checkVersion(this.scenarioDefinition, processor);
log.info("Loading scenarios from {}", this.scenarioDefinition);
final CollectingErrorEventHandler handler = new CollectingErrorEventHandler();
final ConversionService conversionService = new ConversionService();
final Scenarios scenarios = conversionService.readXml(this.scenarioDefinition, Scenarios.class, scenarioSchema, handler);
if (!handler.hasErrors()) {
log.info("Loading scenario content from {}", this.getScenarioRepository());
@ -185,6 +185,9 @@ public class ConfigurationLoader {
s.setSchema(repository.createSchema(def));
s.setSchematronValidations(repository.createSchematronTransformations(def));
s.setReportTransformation(repository.createReportTransformation(def));
s.setFactory(repository.getResolvingConfigurationStrategy());
s.setUriResolver(repository.getResolver());
s.setUnparsedTextURIResolver(repository.getUnparsedTextURIResolver());
if (def.getAcceptMatch() != null) {
s.setAcceptExecutable(repository.createAccepptExecutable(def));
}

View file

@ -46,6 +46,9 @@ public class FallbackBuilder implements Builder<Scenario> {
object.setCreateReport(build.getObject().getLeft());
final Scenario s = new Scenario(object);
s.setFallback(true);
s.setFactory(repository.getResolvingConfigurationStrategy());
s.setUriResolver(repository.getResolver());
s.setUnparsedTextURIResolver(repository.getUnparsedTextURIResolver());
s.setReportTransformation(build.getObject().getRight());
result = new Result<>(s);
} else {

View file

@ -88,6 +88,9 @@ public class ScenarioBuilder implements Builder<Scenario> {
buildReport(repository, errors, scenario);
buildAccept(repository, errors, scenario);
buildNamespaces(scenario);
scenario.setFactory(repository.getResolvingConfigurationStrategy());
scenario.setUriResolver(repository.getResolver());
scenario.setUnparsedTextURIResolver(repository.getUnparsedTextURIResolver());
return new Result<>(scenario, errors);
}

View file

@ -21,6 +21,7 @@ import java.io.InputStreamReader;
import java.io.Reader;
import java.io.StringWriter;
import java.net.URI;
import java.util.List;
import java.util.Optional;
import org.apache.commons.io.IOUtils;
@ -44,7 +45,7 @@ import de.kosit.validationtool.model.scenarios.Scenarios;
@RequiredArgsConstructor
class ConfigHandler extends BaseHandler {
private final Configuration configuration;
private final List<Configuration> configuration;
private final ConversionService conversionService;
@ -64,7 +65,8 @@ class ConfigHandler extends BaseHandler {
}
private Optional<String> getSource() {
final URI fileUri = (URI) this.configuration.getAdditionalParameters().get(Keys.SCENARIOS_FILE);
final URI fileUri = (URI) this.configuration.get(0).getAdditionalParameters().get(Keys.SCENARIOS_FILE);
return fileUri != null ? loadFile(fileUri) : loadFromConfig();
}
@ -80,7 +82,7 @@ class ConfigHandler extends BaseHandler {
private Optional<String> loadFromConfig() {
final Optional<String> result;
final Scenarios scenarios = (Scenarios) this.configuration.getAdditionalParameters().get(Keys.SCENARIO_DEFINITION);
final Scenarios scenarios = (Scenarios) this.configuration.get(0).getAdditionalParameters().get(Keys.SCENARIO_DEFINITION);
if (scenarios != null) {
final String s = this.conversionService.writeXml(scenarios);
result = Optional.of(s);

View file

@ -36,6 +36,8 @@ import de.kosit.validationtool.impl.ConversionService;
import de.kosit.validationtool.impl.DefaultCheck;
import de.kosit.validationtool.model.daemon.HealthType;
import net.sf.saxon.s9api.Processor;
/**
* HTTP-Daemon für die Bereitstellung der Prüf-Funktionalität via http.
*
@ -76,17 +78,18 @@ public class Daemon {
*
* @param config the configuration to use
*/
public void startServer(final Configuration config) {
public void startServer(final Processor processor, final Configuration... config) {
HttpServer server = null;
try {
final ConversionService healthConverter = new ConversionService();
healthConverter.initialize(HealthType.class.getPackage());
final ConversionService converter = new ConversionService();
final DefaultCheck check = new DefaultCheck(processor, config);
server = HttpServer.create(getSocket(), 0);
server.createContext("/", createRootHandler(config));
server.createContext("/server/health", new HealthHandler(config, healthConverter));
server.createContext("/server/config", new ConfigHandler(config, converter));
server.createContext("/", createRootHandler(check, processor));
server.createContext("/server/health", new HealthHandler(check.getConfiguration(), healthConverter));
server.createContext("/server/config", new ConfigHandler(check.getConfiguration(), converter));
server.setExecutor(createExecutor());
server.start();
log.info("Server {} started", server.getAddress());
@ -96,10 +99,9 @@ public class Daemon {
}
}
private HttpHandler createRootHandler(final Configuration config) {
private HttpHandler createRootHandler(final DefaultCheck check, final Processor processor) {
final HttpHandler rootHandler;
final DefaultCheck check = new DefaultCheck(config);
final CheckHandler checkHandler = new CheckHandler(check, config.getContentRepository().getProcessor());
final CheckHandler checkHandler = new CheckHandler(check, processor);
if (this.guiEnabled) {
final GuiHandler gui = new GuiHandler();
rootHandler = new RoutingHandler(checkHandler, gui);

View file

@ -17,6 +17,7 @@
package de.kosit.validationtool.daemon;
import java.io.IOException;
import java.util.List;
import com.sun.net.httpserver.HttpExchange;
@ -39,7 +40,7 @@ import de.kosit.validationtool.model.daemon.MemoryType;
@RequiredArgsConstructor
class HealthHandler extends BaseHandler {
private final Configuration scenarios;
private final List<Configuration> scenarios;
private final ConversionService conversionService;
@ -55,7 +56,7 @@ class HealthHandler extends BaseHandler {
final HealthType h = new HealthType();
h.setMemory(createMemory());
h.setApplication(createApplication());
h.setStatus(!this.scenarios.getScenarios().isEmpty() ? "UP" : "DOWN");
h.setStatus(this.scenarios.stream().mapToLong(c -> c.getScenarios().size()).sum() > 0 ? "UP" : "DOWN");
return h;
}

View file

@ -34,7 +34,6 @@ import javax.xml.validation.Schema;
import javax.xml.validation.SchemaFactory;
import org.apache.commons.lang3.StringUtils;
import org.w3c.dom.ls.LSResourceResolver;
import org.xml.sax.SAXException;
import lombok.Getter;
@ -67,8 +66,6 @@ import net.sf.saxon.s9api.XsltExecutable;
@Slf4j
public class ContentRepository {
private Schema reportInputSchema;
@Getter
private final Processor processor;
@ -90,10 +87,10 @@ public class ContentRepository {
* @param strategy the security and resolving strategy
* @param repository the repository.
*/
public ContentRepository(final ResolvingConfigurationStrategy strategy, final URI repository) {
public ContentRepository(final Processor processor, final ResolvingConfigurationStrategy strategy, final URI repository) {
this.repository = repository;
this.resolvingConfigurationStrategy = strategy;
this.processor = this.resolvingConfigurationStrategy.getProcessor();
this.processor = processor;
this.resolver = this.resolvingConfigurationStrategy.createResolver(repository);
this.unparsedTextURIResolver = this.resolvingConfigurationStrategy.createUnparsedTextURIResolver(repository);
this.schemaFactory = this.resolvingConfigurationStrategy.createSchemaFactory();
@ -108,20 +105,15 @@ public class ContentRepository {
}
}
private Schema createSchema(final Source[] schemaSources, final LSResourceResolver resourceResolver) {
private Schema createSchema(final Source[] schemaSources) {
try {
final SchemaFactory sf = this.schemaFactory;
sf.setResourceResolver(resourceResolver);
return sf.newSchema(schemaSources);
this.schemaFactory.setResourceResolver(null);
return this.schemaFactory.newSchema(schemaSources);
} catch (final SAXException e) {
throw new IllegalArgumentException("Can not load schema from sources " + schemaSources[0].getSystemId(), e);
}
}
private Schema createSchema(final Source[] schemaSources) {
return createSchema(schemaSources, null);
}
/**
* Lädt ein XSL von der angegebenen URI
*
@ -158,40 +150,13 @@ public class ContentRepository {
* @return das erzeugte Schema
*/
public Schema createSchema(final URL url) {
return createSchema(url, null);
return createSchema(new Source[] { resolve(url) });
}
public Schema createSchema(final URI uri) {
return createSchema(new Source[] { resolveInRepository(uri) });
}
public Schema createSchema(final URL url, final LSResourceResolver resourceResolver) {
log.info("Load schema from source {}", url.getPath());
return createSchema(new Source[] { resolve(url) }, resourceResolver);
}
/**
* Liefert das definiert Schema für die Szenario-Konfiguration
*
* @return Scenario-Schema
*/
public Schema getScenarioSchema() {
return createSchema(ContentRepository.class.getResource("/xsd/scenarios.xsd"));
}
/**
* Liefert das definierte Schema für die Validierung des [@link CreateReportInput}
*
* @return ReportInput-Schema
*/
public Schema getReportInputSchema() {
if (this.reportInputSchema == null) {
final Source source = resolve(ContentRepository.class.getResource("/xsd/createReportInput.xsd"));
this.reportInputSchema = createSchema(new Source[] { source }, new ClassPathResourceResolver("/xsd"));
}
return this.reportInputSchema;
}
/**
* Erzeugt ein Schema auf Basis der übegebenen URIs
*

View file

@ -19,6 +19,7 @@ package de.kosit.validationtool.impl;
import static de.kosit.validationtool.impl.DateFactory.createTimestamp;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.Map;
@ -42,6 +43,7 @@ import de.kosit.validationtool.impl.tasks.ScenarioSelectionAction;
import de.kosit.validationtool.impl.tasks.SchemaValidationAction;
import de.kosit.validationtool.impl.tasks.SchematronValidationAction;
import de.kosit.validationtool.impl.tasks.ValidateReportInputAction;
import de.kosit.validationtool.impl.xml.ProcessorProvider;
import de.kosit.validationtool.model.reportInput.CreateReportInput;
import de.kosit.validationtool.model.reportInput.EngineType;
import de.kosit.validationtool.model.reportInput.XMLSyntaxError;
@ -61,31 +63,36 @@ public class DefaultCheck implements Check {
private final ConversionService conversionService;
@Getter
private final Configuration configuration;
private final List<Configuration> configuration;
@Getter
private final List<CheckAction> checkSteps;
@Getter
private final Processor processor;
public DefaultCheck(final Configuration... configuration) {
this(ProcessorProvider.getProcessor(), configuration);
}
/**
* Creates a new instance for the {@link Configuration}.
*
* @param configuration the Configuration
*/
public DefaultCheck(final Configuration configuration) {
this.configuration = configuration;
final ContentRepository content = configuration.getContentRepository();
final Processor processor = content.getProcessor();
public DefaultCheck(final Processor processor, final Configuration... configuration) {
this.configuration = Arrays.asList(configuration);
this.processor = processor;
this.conversionService = new ConversionService();
this.checkSteps = new ArrayList<>();
this.checkSteps.add(new DocumentParseAction(processor));
this.checkSteps.add(new CreateDocumentIdentificationAction());
this.checkSteps.add(new ScenarioSelectionAction(new ScenarioRepository(configuration)));
this.checkSteps.add(new SchemaValidationAction(content.getResolvingConfigurationStrategy(), processor));
this.checkSteps.add(new SchematronValidationAction(content.getResolver(), this.conversionService));
this.checkSteps.add(new ValidateReportInputAction(this.conversionService, content.getReportInputSchema()));
this.checkSteps.add(
new CreateReportAction(processor, this.conversionService, content.getResolver(), content.getUnparsedTextURIResolver()));
this.checkSteps.add(new SchemaValidationAction(processor));
this.checkSteps.add(new SchematronValidationAction(this.conversionService));
this.checkSteps.add(new ValidateReportInputAction(this.conversionService, SchemaProvider.getReportInputSchema()));
this.checkSteps.add(new CreateReportAction(processor, this.conversionService));
this.checkSteps.add(new ComputeAcceptanceAction());
}
@ -125,8 +132,7 @@ public class DefaultCheck implements Check {
}
private Result createResult(final Bag t) {
final DefaultResult result = new DefaultResult(t.getReport(), t.getAcceptStatus(),
new HtmlExtractor(this.configuration.getContentRepository().getProcessor()));
final DefaultResult result = new DefaultResult(t.getReport(), t.getAcceptStatus(), new HtmlExtractor(this.processor));
result.setWellformed(t.getParserResult().isValid());
result.setReportInput(t.getReportInput());
if (t.getSchemaValidationResult() != null) {

View file

@ -50,4 +50,20 @@ public class Printer {
public static void writeErr(final String message, final Object... params) {
System.err.println(new MessageFormat(message, Locale.ENGLISH).format(params));
}
/**
* Writes to standard error channel and prints a stacktrace.
*
* @param ex the exception
* @param message the message with placeholders
* @param params the params
*/
@SuppressWarnings("squid:S1148")
public static void writeErr(final Exception ex, final String message, final Object... params) {
writeErr(message, params);
if (ex != null) {
ex.printStackTrace();
}
}
}

View file

@ -20,6 +20,7 @@ import java.util.Collections;
import java.util.List;
import java.util.Optional;
import javax.xml.transform.URIResolver;
import javax.xml.validation.Schema;
import lombok.AllArgsConstructor;
@ -27,9 +28,11 @@ import lombok.Getter;
import lombok.RequiredArgsConstructor;
import lombok.Setter;
import de.kosit.validationtool.api.ResolvingConfigurationStrategy;
import de.kosit.validationtool.model.scenarios.ResourceType;
import de.kosit.validationtool.model.scenarios.ScenarioType;
import net.sf.saxon.lib.UnparsedTextURIResolver;
import net.sf.saxon.s9api.XPathExecutable;
import net.sf.saxon.s9api.XPathSelector;
import net.sf.saxon.s9api.XsltExecutable;
@ -65,6 +68,12 @@ public class Scenario {
private XPathExecutable acceptExecutable;
private ResolvingConfigurationStrategy factory;
private URIResolver uriResolver;
private UnparsedTextURIResolver unparsedTextURIResolver;
@Setter
private List<Transformation> schematronValidations;

View file

@ -16,6 +16,7 @@
package de.kosit.validationtool.impl;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.stream.Collectors;
@ -38,20 +39,30 @@ import net.sf.saxon.s9api.XdmNode;
public class ScenarioRepository {
private final Configuration configuration;
public static final String DEFAULT = "default";
public ScenarioRepository(final Configuration configuration) {
this.configuration = configuration;
log.info("Loaded scenarios for {} by {} from {}. The following scenarios are available:\n\n{}", configuration.getName(),
configuration.getAuthor(), configuration.getDate(), summarizeScenarios());
public static final String DEFAULT_ID = DEFAULT + "_1";
private final List<Configuration> configuration;
public ScenarioRepository(final Configuration... configuration) {
if (configuration.length == 0) {
throw new IllegalArgumentException("Must provide at least one configuration");
}
this.configuration = Arrays.asList(configuration);
this.configuration.forEach(v -> log.info("Loaded scenarios for {} by {} from {}.", v.getName(), v.getAuthor(), v.getDate()));
log.info("The following scenarios are available:\n{}", summarizeScenarios());
}
public Scenario getFallbackScenario() {
return this.configuration.getFallbackScenario();
if (this.configuration.size() > 1) {
log.warn("Multiple configurations found. Using fallback scenario from first configuration");
}
return this.configuration.get(0).getFallbackScenario();
}
public List<Scenario> getScenarios() {
return this.configuration.getScenarios();
return this.configuration.stream().flatMap(c -> c.getScenarios().stream()).collect(Collectors.toList());
}
private String summarizeScenarios() {

View file

@ -0,0 +1,87 @@
/*
* Copyright 2017-2020 Koordinierungsstelle für IT-Standards (KoSIT)
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package de.kosit.validationtool.impl;
import java.io.IOException;
import java.net.URISyntaxException;
import java.net.URL;
import javax.xml.transform.Source;
import javax.xml.transform.stream.StreamSource;
import javax.xml.validation.Schema;
import javax.xml.validation.SchemaFactory;
import org.w3c.dom.ls.LSResourceResolver;
import org.xml.sax.SAXException;
import de.kosit.validationtool.impl.xml.ClassPathResourceResolver;
/**
* @author Andreas Penski
*/
public class SchemaProvider {
private static Schema reportInputSchema;
/**
* Liefert das definierte Schema für die Validierung des [@link CreateReportInput}
*
* @return ReportInput-Schema
*/
public static Schema getReportInputSchema() {
if (reportInputSchema == null) {
final SchemaFactory sf = ResolvingMode.STRICT_RELATIVE.getStrategy().createSchemaFactory();
final Source source = resolve(SchemaProvider.class.getResource("/xsd/createReportInput.xsd"));
reportInputSchema = createSchema(sf, new Source[] { source }, new ClassPathResourceResolver("/xsd"));
}
return reportInputSchema;
}
private static Schema createSchema(final SchemaFactory sf, final Source[] schemaSources, final LSResourceResolver resourceResolver) {
try {
sf.setResourceResolver(resourceResolver);
return sf.newSchema(schemaSources);
} catch (final SAXException e) {
throw new IllegalArgumentException("Can not load schema from sources " + schemaSources[0].getSystemId(), e);
}
}
private static Schema createSchema(final SchemaFactory sf, final Source... schemaSources) {
return createSchema(sf, schemaSources, null);
}
@SuppressWarnings("java:S2095") // xml stack requires not closing the resource here
private static Source resolve(final URL resource) {
try {
final String rawPath = resource.toURI().getRawPath();
return new StreamSource(resource.openStream(), rawPath);
} catch (final IOException | URISyntaxException e) {
throw new IllegalStateException("Can not load schema for resource " + resource.getPath(), e);
}
}
/**
* Liefert das definiert Schema für die Szenario-Konfiguration
*
* @return Scenario-Schema
*/
public static Schema getScenarioSchema() {
final SchemaFactory sf = ResolvingMode.STRICT_RELATIVE.getStrategy().createSchemaFactory();
return createSchema(sf, resolve(SchemaProvider.class.getResource("/xsd/scenarios.xsd")));
}
}

View file

@ -23,7 +23,6 @@ import java.util.stream.Collectors;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Marshaller;
import javax.xml.bind.util.JAXBSource;
import javax.xml.transform.URIResolver;
import org.xml.sax.ContentHandler;
import org.xml.sax.DTDHandler;
@ -45,7 +44,6 @@ import de.kosit.validationtool.impl.EngineInformation;
import de.kosit.validationtool.impl.Scenario;
import de.kosit.validationtool.model.reportInput.XMLSyntaxError;
import net.sf.saxon.lib.UnparsedTextURIResolver;
import net.sf.saxon.s9api.BuildingContentHandler;
import net.sf.saxon.s9api.DocumentBuilder;
import net.sf.saxon.s9api.Processor;
@ -172,10 +170,6 @@ public class CreateReportAction implements CheckAction {
private final ConversionService conversionService;
private final URIResolver resolver;
private final UnparsedTextURIResolver unparsedTextURIResolver;
private static XsltExecutable loadFromScenario(final Scenario object) {
return object.getReportTransformation().getExecutable();
}
@ -198,9 +192,11 @@ public class CreateReportAction implements CheckAction {
transformer.setInitialContextNode(root);
final CollectingErrorEventHandler e = new CollectingErrorEventHandler();
transformer.setMessageListener(e);
transformer.setURIResolver(this.resolver);
if (this.unparsedTextURIResolver != null) {
transformer.getUnderlyingController().setUnparsedTextURIResolver(this.unparsedTextURIResolver);
final Scenario scenario = results.getScenarioSelectionResult().getObject();
transformer.setURIResolver(scenario.getUriResolver());
if (scenario.getUnparsedTextURIResolver() != null) {
transformer.getUnderlyingController().setUnparsedTextURIResolver(scenario.getUnparsedTextURIResolver());
}
if (parsedDocument != null) {
transformer.setParameter(new QName("input-document"), parsedDocument);

View file

@ -38,7 +38,6 @@ import lombok.Setter;
import lombok.extern.slf4j.Slf4j;
import de.kosit.validationtool.api.Input;
import de.kosit.validationtool.api.ResolvingConfigurationStrategy;
import de.kosit.validationtool.impl.CollectingErrorEventHandler;
import de.kosit.validationtool.impl.Scenario;
import de.kosit.validationtool.impl.input.AbstractInput;
@ -134,8 +133,6 @@ public class SchemaValidationAction implements CheckAction {
private static final String LIMIT_PARAMETER = "schema.validation.inmem.limit";
private final ResolvingConfigurationStrategy factory;
private final Processor processor;
@Setter(AccessLevel.PACKAGE)
@ -147,7 +144,7 @@ public class SchemaValidationAction implements CheckAction {
final CollectingErrorEventHandler errorHandler = new CollectingErrorEventHandler();
try ( final SourceProvider validateInput = resolveSource(results) ) {
final Validator validator = this.factory.createValidator(scenario.getSchema());
final Validator validator = scenario.getFactory().createValidator(scenario.getSchema());
validator.setErrorHandler(errorHandler);
validator.validate(validateInput.getSource());
return new Result<>(!errorHandler.hasErrors(), errorHandler.getErrors());

View file

@ -19,7 +19,6 @@ package de.kosit.validationtool.impl.tasks;
import java.util.List;
import java.util.stream.Collectors;
import javax.xml.transform.URIResolver;
import javax.xml.transform.dom.DOMSource;
import org.oclc.purl.dsdl.svrl.SchematronOutput;
@ -30,6 +29,7 @@ import lombok.extern.slf4j.Slf4j;
import de.kosit.validationtool.impl.CollectingErrorEventHandler;
import de.kosit.validationtool.impl.ConversionService;
import de.kosit.validationtool.impl.Scenario;
import de.kosit.validationtool.impl.Scenario.Transformation;
import de.kosit.validationtool.model.reportInput.CreateReportInput;
import de.kosit.validationtool.model.reportInput.ValidationResultsSchematron;
import de.kosit.validationtool.model.reportInput.ValidationResultsSchematron.Results;
@ -49,21 +49,20 @@ import net.sf.saxon.s9api.XsltTransformer;
@Slf4j
public class SchematronValidationAction implements CheckAction {
private final URIResolver resolver;
private final ConversionService conversionService;
private List<ValidationResultsSchematron> validate(final Bag results, final XdmNode document, final Scenario scenario) {
return scenario.getSchematronValidations().stream().map(v -> validate(results, document, v)).collect(Collectors.toList());
return scenario.getSchematronValidations().stream().map(v -> validate(scenario, results, document, v)).collect(Collectors.toList());
}
private ValidationResultsSchematron validate(final Bag results, final XdmNode document, final Scenario.Transformation validation) {
private ValidationResultsSchematron validate(final Scenario scenario, final Bag results, final XdmNode document,
final Transformation validation) {
final ValidationResultsSchematron s = new ValidationResultsSchematron();
s.setResource(validation.getResourceType());
try {
final XsltTransformer transformer = validation.getExecutable().load();
// resolving nur relative zum Repository
transformer.setURIResolver(this.resolver);
transformer.setURIResolver(scenario.getUriResolver());
final CollectingErrorEventHandler e = new CollectingErrorEventHandler();
transformer.setMessageListener(e);

View file

@ -28,34 +28,14 @@ import lombok.extern.slf4j.Slf4j;
import de.kosit.validationtool.api.ResolvingConfigurationStrategy;
import net.sf.saxon.s9api.Processor;
/**
* @author Andreas Penski
*/
@Slf4j
public abstract class BaseResolvingStrategy implements ResolvingConfigurationStrategy {
protected static final String DISSALLOW_DOCTYPE_DECL_FEATURE = "http://apache.org/xml/features/disallow-doctype-decl";
protected static final String LOAD_EXTERNAL_DTD_FEATURE = "http://apache.org/xml/features/nonvalidating/load-external-dtd";
protected static final String FEATURE_SECURE_PROCESSING = "http://javax.xml.XMLConstants/feature/secure-processing";
private static final String ORACLE_XERCES_CLASS = "com.sun.org.apache.xerces.internal.impl.Constants";
private Processor processor;
@Override
public Processor getProcessor() {
if (this.processor == null) {
this.processor = createProcessor();
}
return this.processor;
}
protected abstract Processor createProcessor();
public static void forceOpenJdkXmlImplementation() {
if (!isOpenJdkXmlImplementationAvailable()) {
throw new IllegalStateException("No OpenJDK version of XERCES found");

View file

@ -14,7 +14,7 @@
* limitations under the License.
*/
package de.kosit.validationtool.impl;
package de.kosit.validationtool.impl.xml;
import java.io.IOException;
import java.io.InputStream;
@ -31,15 +31,13 @@ import lombok.RequiredArgsConstructor;
import lombok.Setter;
import lombok.extern.slf4j.Slf4j;
import de.kosit.validationtool.impl.xml.RelativeUriResolver;
/**
* {@link LSResourceResolver} der objekte relativ zu einem Basis-Pfad aus dem Classpath der Anwendung laden kann.
*
* @author Andreas Penski
*/
@Slf4j
class ClassPathResourceResolver implements LSResourceResolver {
public class ClassPathResourceResolver implements LSResourceResolver {
/**
* Simple {@link LSInput}-Implementierung, die einen Stream liefern kann

View file

@ -0,0 +1,117 @@
/*
* Copyright 2017-2020 Koordinierungsstelle für IT-Standards (KoSIT)
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package de.kosit.validationtool.impl.xml;
import java.io.Reader;
import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import javax.xml.XMLConstants;
import javax.xml.transform.Result;
import javax.xml.transform.TransformerException;
import lombok.SneakyThrows;
import net.sf.saxon.Configuration;
import net.sf.saxon.expr.XPathContext;
import net.sf.saxon.lib.CollectionFinder;
import net.sf.saxon.lib.Feature;
import net.sf.saxon.lib.FeatureKeys;
import net.sf.saxon.lib.OutputURIResolver;
import net.sf.saxon.lib.ResourceCollection;
import net.sf.saxon.lib.UnparsedTextURIResolver;
import net.sf.saxon.s9api.Processor;
import net.sf.saxon.trans.XPathException;
/**
* @author Andreas Penski
*/
public class ProcessorProvider {
private static class SecureUriResolver implements CollectionFinder, OutputURIResolver, UnparsedTextURIResolver {
public static final String MESSAGE = "Configuration error. Resolving ist not allowed";
@Override
public OutputURIResolver newInstance() {
return this;
}
@Override
public Result resolve(final String href, final String base) throws TransformerException {
throw new IllegalStateException(MESSAGE);
}
@Override
public void close(final Result result) throws TransformerException {
throw new IllegalStateException(MESSAGE);
}
@Override
public Reader resolve(final URI absoluteURI, final String encoding, final Configuration config) throws XPathException {
throw new IllegalStateException(MESSAGE);
}
@Override
public ResourceCollection findCollection(final XPathContext context, final String collectionURI) throws XPathException {
throw new IllegalStateException(MESSAGE);
}
}
protected static final String DISSALLOW_DOCTYPE_DECL_FEATURE = "http://apache.org/xml/features/disallow-doctype-decl";
protected static final String LOAD_EXTERNAL_DTD_FEATURE = "http://apache.org/xml/features/nonvalidating/load-external-dtd";
protected static final String FEATURE_SECURE_PROCESSING = "http://javax.xml.XMLConstants/feature/secure-processing";
private static Processor processor;
@SneakyThrows
private static String encode(final String input) {
return URLEncoder.encode(input, StandardCharsets.UTF_8.name());
}
public static Processor getProcessor() {
if (processor == null) {
processor = createProcessor();
}
return processor;
}
private static Processor createProcessor() {
final Processor processor = new Processor(false);
// verhindere global im Prinzip alle resolving strategien
final SecureUriResolver resolver = new SecureUriResolver();
processor.getUnderlyingConfiguration().setCollectionFinder(resolver);
processor.getUnderlyingConfiguration().setOutputURIResolver(resolver);// NOSONAR
processor.getUnderlyingConfiguration().setUnparsedTextURIResolver(resolver);
// grundsätzlich Feature-konfiguration:
processor.setConfigurationProperty(Feature.DTD_VALIDATION, false);
processor.setConfigurationProperty(Feature.ENTITY_RESOLVER_CLASS, "");
processor.setConfigurationProperty(Feature.XINCLUDE, false);
processor.setConfigurationProperty(Feature.ALLOW_EXTERNAL_FUNCTIONS, false);
// Konfiguration des zu verwendenden Parsers, wenn Saxon selbst einen erzeugen muss, bspw. beim XSL parsen
processor.setConfigurationProperty(FeatureKeys.XML_PARSER_FEATURE + encode(FEATURE_SECURE_PROCESSING), true); // NOSONAR
processor.setConfigurationProperty(FeatureKeys.XML_PARSER_FEATURE + encode(DISSALLOW_DOCTYPE_DECL_FEATURE), true);// NOSONAR
processor.setConfigurationProperty(FeatureKeys.XML_PARSER_FEATURE + encode(LOAD_EXTERNAL_DTD_FEATURE), false);// NOSONAR
processor.setConfigurationProperty(FeatureKeys.XML_PARSER_FEATURE + encode(XMLConstants.ACCESS_EXTERNAL_DTD), false);// NOSONAR
return processor;
}
}

View file

@ -16,32 +16,17 @@
package de.kosit.validationtool.impl.xml;
import java.io.Reader;
import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import javax.xml.XMLConstants;
import javax.xml.transform.Result;
import javax.xml.transform.TransformerException;
import javax.xml.transform.URIResolver;
import javax.xml.validation.Schema;
import javax.xml.validation.SchemaFactory;
import javax.xml.validation.Validator;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import net.sf.saxon.Configuration;
import net.sf.saxon.expr.XPathContext;
import net.sf.saxon.lib.CollectionFinder;
import net.sf.saxon.lib.Feature;
import net.sf.saxon.lib.FeatureKeys;
import net.sf.saxon.lib.OutputURIResolver;
import net.sf.saxon.lib.ResourceCollection;
import net.sf.saxon.lib.UnparsedTextURIResolver;
import net.sf.saxon.s9api.Processor;
import net.sf.saxon.trans.XPathException;
/**
* @author Andreas Penski
@ -49,78 +34,16 @@ import net.sf.saxon.trans.XPathException;
@RequiredArgsConstructor
public class StrictRelativeResolvingStrategy extends BaseResolvingStrategy {
private static class SecureUriResolver implements CollectionFinder, OutputURIResolver, UnparsedTextURIResolver {
public static final String MESSAGE = "Configuration error. Resolving ist not allowed";
@Override
public OutputURIResolver newInstance() {
return this;
}
@Override
public Result resolve(final String href, final String base) throws TransformerException {
throw new IllegalStateException(MESSAGE);
}
@Override
public void close(final Result result) throws TransformerException {
throw new IllegalStateException(MESSAGE);
}
@Override
public Reader resolve(final URI absoluteURI, final String encoding, final Configuration config) throws XPathException {
throw new IllegalStateException(MESSAGE);
}
@Override
public ResourceCollection findCollection(final XPathContext context, final String collectionURI) throws XPathException {
throw new IllegalStateException(MESSAGE);
}
}
/**
* e.g. don't allow any scheme
*/
private static final String EMPTY_SCHEME = "";
@Override
public SchemaFactory createSchemaFactory() {
forceOpenJdkXmlImplementation();
@SuppressWarnings("java:S2755") //
final SchemaFactory sf = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
disableExternalEntities(sf);
allowExternalSchema(sf, "file");
return sf;
}
@Override
protected Processor createProcessor() {
final Processor processor = new Processor(false);
// verhindere global im Prinzip alle resolving strategien
final SecureUriResolver resolver = new SecureUriResolver();
processor.getUnderlyingConfiguration().setCollectionFinder(resolver);
processor.getUnderlyingConfiguration().setOutputURIResolver(resolver);// NOSONAR
processor.getUnderlyingConfiguration().setUnparsedTextURIResolver(resolver);
// grundsätzlich Feature-konfiguration:
processor.setConfigurationProperty(Feature.DTD_VALIDATION, false);
processor.setConfigurationProperty(Feature.ENTITY_RESOLVER_CLASS, "");
processor.setConfigurationProperty(Feature.XINCLUDE, false);
processor.setConfigurationProperty(Feature.ALLOW_EXTERNAL_FUNCTIONS, false);
// Konfiguration des zu verwendenden Parsers, wenn Saxon selbst einen erzeugen muss, bspw. beim XSL parsen
processor.setConfigurationProperty(FeatureKeys.XML_PARSER_FEATURE + encode(FEATURE_SECURE_PROCESSING), true); // NOSONAR
processor.setConfigurationProperty(FeatureKeys.XML_PARSER_FEATURE + encode(DISSALLOW_DOCTYPE_DECL_FEATURE), true);// NOSONAR
processor.setConfigurationProperty(FeatureKeys.XML_PARSER_FEATURE + encode(LOAD_EXTERNAL_DTD_FEATURE), false);// NOSONAR
processor.setConfigurationProperty(FeatureKeys.XML_PARSER_FEATURE + encode(XMLConstants.ACCESS_EXTERNAL_DTD), false);// NOSONAR
return processor;
}
@SneakyThrows
private static String encode(final String input) {
return URLEncoder.encode(input, StandardCharsets.UTF_8.name());
}
@Override
public URIResolver createResolver(final URI repositoryURI) {
return new RelativeUriResolver(repositoryURI);

View file

@ -114,22 +114,22 @@ public class CommandLine {
}
public static String getOutput() {
return new String(out.getOut().toByteArray());
return out.getOut().toString();
}
public static String getErrorOutput() {
return new String(error.getOut().toByteArray());
return error.getOut().toString();
}
public List<String> getOutputLines() {
public static List<String> getOutputLines() {
return readLines(out.getOut().toByteArray());
}
public List<String> getErrorLines() {
public static List<String> getErrorLines() {
return readLines(error.getOut().toByteArray());
}
private List<String> readLines(final byte[] bytes) {
private static List<String> readLines(final byte[] bytes) {
try ( final ByteArrayInputStream in = new ByteArrayInputStream(bytes);
final Reader r = new InputStreamReader(in) ) {

View file

@ -57,6 +57,7 @@ public class CommandlineApplicationTest {
if (Files.exists(this.output)) {
FileUtils.deleteDirectory(this.output.toFile());
}
TypeConverter.counter.clear();
}
@After
@ -76,20 +77,20 @@ public class CommandlineApplicationTest {
final String[] args = new String[] { "-?" };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getErrorOutput()).isEmpty();
checkForHelp(this.commandLine.getOutputLines());
checkForHelp(CommandLine.getOutputLines());
}
private static void checkForHelp(final List<String> outputLines) {
assertThat(outputLines.size()).isGreaterThan(0);
outputLines.subList(1, outputLines.size() - 1).forEach(l -> assertThat(l.startsWith(" -") || l.startsWith(" ")));
assertThat(outputLines.size()).isPositive();
assertThat(outputLines.stream().filter(l -> l.startsWith("Usage: KoSIT Validator"))).hasSize(1);
}
@Test
public void testRequiredScenarioFile() {
final String[] args = new String[] { "-d", "arguments", "egal welche", "argument drin sind" };
final String[] args = new String[] { "arguments", "egal welche", "argumente drin sind" };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getErrorOutput()).isNotEmpty();
assertThat(CommandLine.getErrorOutput()).contains("Missing required option: s");
assertThat(CommandLine.getErrorOutput()).contains("Missing required option: '--scenarios");
}
@Test
@ -102,10 +103,11 @@ public class CommandlineApplicationTest {
@Test
public void testIncorrectRepository() {
final String[] args = new String[] { "-s", Paths.get(Simple.SCENARIOS).toString(), Paths.get(Simple.NOT_EXISTING).toString() };
final String[] args = new String[] { "-s", Paths.get(Simple.SCENARIOS).toString(), "-r", Paths.get(Simple.NOT_EXISTING).toString(),
Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getErrorOutput()).isNotEmpty();
assertThat(CommandLine.getErrorOutput()).contains("Can not resolve");
assertThat(CommandLine.getErrorOutput()).contains("Not a valid path for repository");
}
@Test
@ -168,7 +170,7 @@ public class CommandlineApplicationTest {
final String[] args = new String[] { "-s", Paths.get(Simple.SCENARIOS).toString(), "-r",
Paths.get(Simple.REPOSITORY_URI).toString(), };
CommandLineApplication.mainProgram(args);
checkForHelp(this.commandLine.getOutputLines());
checkForHelp(CommandLine.getOutputLines());
}
@Test
@ -178,7 +180,7 @@ public class CommandlineApplicationTest {
Paths.get(Simple.REPOSITORY_URI).toString(), "-o", this.output.toString(), Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getErrorOutput()).contains(RESULT_OUTPUT);
assertThat(this.commandLine.getOutputLines()).haveAtLeastOne(new Condition<>(
assertThat(CommandLine.getOutputLines()).haveAtLeastOne(new Condition<>(
s -> StringUtils.contains(s, "<?xml version=\"1.0\" " + "encoding=\"UTF-8\"?>"), "Must " + "contain xml preambel"));
}
@ -189,7 +191,7 @@ public class CommandlineApplicationTest {
Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getErrorOutput()).contains(RESULT_OUTPUT);
assertThat(Files.list(this.output).filter(f -> f.toString().endsWith(".html")).count()).isGreaterThan(0);
assertThat(Files.list(this.output).filter(f -> f.toString().endsWith(".html")).count()).isPositive();
}
@Test
@ -227,4 +229,76 @@ public class CommandlineApplicationTest {
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getErrorOutput()).contains(RESULT_OUTPUT);
}
@Test
public void testUnexpectedDaemonFlag() {
final String[] args = new String[] { "-D", "-s", Paths.get(Simple.SCENARIOS).toString(), "-r",
Paths.get(Simple.REPOSITORY_URI).toString(), Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getErrorOutput()).contains("Will ignore cli mode options");
}
@Test
public void testParsingError() {
final String[] args = new String[] { "-s", "-r", Paths.get(Simple.REPOSITORY_URI).toString(),
Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getErrorOutput()).contains("Expected parameter for option");
}
@Test
public void loadMultipleScenarios() {
final String[] args = new String[] { "-s", "s1=" + Paths.get(Simple.SCENARIOS).toString(), "-s",
"s2=" + Paths.get(Simple.OTHER_SCENARIOS).toString(), "-r", "s1=" + Paths.get(Simple.REPOSITORY_URI).toString(), "-r",
"s2=" + Paths.get(Simple.REPOSITORY_URI).toString(), Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getOutput()).contains("Processing of 1 objects completed");
}
@Test
public void loadMultipleScenariosSingleRepository() {
final String[] args = new String[] { "-s", "s1=" + Paths.get(Simple.SCENARIOS).toString(), "-s",
"s2=" + Paths.get(Simple.OTHER_SCENARIOS).toString(), "-r", Paths.get(Simple.REPOSITORY_URI).toString(),
Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getOutput()).contains("Processing of 1 objects completed");
}
@Test
public void loadMultipleScenariosMissingRepository() {
final String[] args = new String[] { "-s", "s1=" + Paths.get(Simple.SCENARIOS).toString(), "-s",
"s2=" + Paths.get(Simple.OTHER_SCENARIOS).toString(), "-r", "s1=" + Paths.get(Simple.REPOSITORY_URI).toString(), "-r",
"typo=" + Paths.get(Simple.REPOSITORY_URI).toString(), Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getErrorOutput()).contains("No repository location for scenario definition 's2' specified");
}
@Test
public void loadMultipleOrderedScenarios() {
final String[] args = new String[] { "-s", Paths.get(Simple.SCENARIOS).toString(), "-s",
Paths.get(Simple.OTHER_SCENARIOS).toString(), "-r", Paths.get(Simple.REPOSITORY_URI).toString(), "-r",
Paths.get(Simple.REPOSITORY_URI).toString(), Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getOutput()).contains("Processing of 1 objects completed");
}
@Test
public void checkUnusedRepository() {
final String[] args = new String[] { "-s", "s1=" + Paths.get(Simple.SCENARIOS).toString(), "-r",
"s1=" + Paths.get(Simple.REPOSITORY_URI).toString(), "-r", "unused=" + Paths.get(Simple.REPOSITORY_URI).toString(),
Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getOutput()).contains("Processing of 1 objects completed");
assertThat(CommandLine.getErrorOutput()).contains("Warning: repository definition \"unused\" is not used");
}
@Test
public void checkDuplicationScenarioDefinition() {
final String[] args = new String[] { "-s", "s1=" + Paths.get(Simple.SCENARIOS).toString(), "-r",
"s1=" + Paths.get(Simple.REPOSITORY_URI).toString(), "-r", "unused=" + Paths.get(Simple.REPOSITORY_URI).toString(),
Paths.get(Simple.SIMPLE_VALID).toString() };
CommandLineApplication.mainProgram(args);
assertThat(CommandLine.getOutput()).contains("Processing of 1 objects completed");
assertThat(CommandLine.getErrorOutput()).contains("Warning: repository definition \"unused\" is not used");
}
}

View file

@ -30,6 +30,8 @@ import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;
import de.kosit.validationtool.impl.Helper;
/**
* Test {@link ConfigurationBuilder}.
*
@ -45,7 +47,7 @@ public class ConfigurationBuilderTest {
@Test
public void testNoConfiguration() {
this.exceptions.expect(IllegalStateException.class);
new ConfigurationBuilder().build();
new ConfigurationBuilder().build(Helper.getTestProcessor());
}
@Test
@ -54,7 +56,7 @@ public class ConfigurationBuilderTest {
this.exceptions.expectMessage(Matchers.containsString("fallback"));
final ConfigurationBuilder builder = createSimpleConfiguration();
builder.with((FallbackBuilder) null);
builder.build();
builder.build(Helper.getTestProcessor());
}
@Test
@ -63,7 +65,7 @@ public class ConfigurationBuilderTest {
this.exceptions.expectMessage(Matchers.containsString("schema"));
final ConfigurationBuilder builder = createSimpleConfiguration();
builder.getScenarios().get(0).validate((SchemaBuilder) null);
builder.build();
builder.build(Helper.getTestProcessor());
}
@Test
@ -72,7 +74,7 @@ public class ConfigurationBuilderTest {
this.exceptions.expectMessage(Matchers.containsString("schematron"));
final ConfigurationBuilder builder = createSimpleConfiguration();
builder.getScenarios().get(0).validate(schematron("invalid").source(URI.create("DoesNotExist")));
builder.build();
builder.build(Helper.getTestProcessor());
}
@Test
@ -81,7 +83,7 @@ public class ConfigurationBuilderTest {
this.exceptions.expectMessage(Matchers.containsString("schematron"));
final ConfigurationBuilder builder = createSimpleConfiguration();
builder.getScenarios().get(0).validate(schematron("invalid"));
builder.build();
builder.build(Helper.getTestProcessor());
}
@Test
@ -90,15 +92,18 @@ public class ConfigurationBuilderTest {
this.exceptions.expectMessage(Matchers.containsString("report"));
final ConfigurationBuilder builder = createSimpleConfiguration();
builder.getScenarios().get(0).with(report("invalid"));
builder.build();
builder.build(Helper.getTestProcessor());
}
@Test
public void testDate() {
assertThat(createSimpleConfiguration().date(EPOCH).build().getDate()).isEqualTo("1970-01-01");
assertThat(createSimpleConfiguration().date(new Date(EPOCH.toEpochDay())).build().getDate()).isEqualTo("1970-01-01");
assertThat(createSimpleConfiguration().date((Date) null).build().getDate()).isEqualTo(LocalDate.now().toString());
assertThat(createSimpleConfiguration().date((LocalDate) null).build().getDate()).isEqualTo(LocalDate.now().toString());
assertThat(createSimpleConfiguration().date(EPOCH).build(Helper.getTestProcessor()).getDate()).isEqualTo("1970-01-01");
assertThat(createSimpleConfiguration().date(new Date(EPOCH.toEpochDay())).build(Helper.getTestProcessor()).getDate())
.isEqualTo("1970-01-01");
assertThat(createSimpleConfiguration().date((Date) null).build(Helper.getTestProcessor()).getDate())
.isEqualTo(LocalDate.now().toString());
assertThat(createSimpleConfiguration().date((LocalDate) null).build(Helper.getTestProcessor()).getDate())
.isEqualTo(LocalDate.now().toString());
}
}

View file

@ -21,6 +21,7 @@ import static org.assertj.core.api.Assertions.assertThat;
import org.junit.Test;
import de.kosit.validationtool.api.Configuration;
import de.kosit.validationtool.impl.Helper;
import de.kosit.validationtool.impl.ResolvingMode;
import de.kosit.validationtool.impl.xml.RemoteResolvingStrategy;
import de.kosit.validationtool.impl.xml.StrictRelativeResolvingStrategy;
@ -35,7 +36,7 @@ public class ConfigurationLoaderTest {
final ConfigurationLoader loader = TestConfigurationFactory.loadSimpleConfiguration();
loader.setResolvingStrategy(new StrictRelativeResolvingStrategy());
loader.setResolvingMode(ResolvingMode.ALLOW_REMOTE);
final Configuration config = loader.build();
final Configuration config = loader.build(Helper.getTestProcessor());
assertThat(config.getContentRepository().getResolvingConfigurationStrategy()).isNotInstanceOf(RemoteResolvingStrategy.class);
}
}

View file

@ -25,6 +25,7 @@ import de.kosit.validationtool.api.Configuration;
import de.kosit.validationtool.api.InputFactory;
import de.kosit.validationtool.api.Result;
import de.kosit.validationtool.impl.DefaultCheck;
import de.kosit.validationtool.impl.Helper;
import de.kosit.validationtool.impl.Helper.Simple;
/**
@ -35,7 +36,7 @@ public class SimpleConfigTest {
@Test
public void testSimpleWithApi() {
//@formatter:off
final Configuration config = createSimpleConfiguration().build();
final Configuration config = createSimpleConfiguration().build(Helper.getTestProcessor());
//@formatter:on
final DefaultCheck check = new DefaultCheck(config);
final Result result = check.checkInput(InputFactory.read(Simple.SIMPLE_VALID));

View file

@ -27,6 +27,7 @@ import org.apache.commons.io.IOUtils;
import org.apache.http.HttpStatus;
import org.junit.Test;
import de.kosit.validationtool.impl.Helper;
import de.kosit.validationtool.impl.Helper.Simple;
import io.restassured.builder.MultiPartSpecBuilder;
@ -81,4 +82,11 @@ public class CheckHandlerIT extends BaseIT {
}
}
@Test
public void testLarge() throws IOException {
try ( final InputStream io = Helper.LARGE_XML.toURL().openStream() ) {
given().contentType(APPLICATION_XML).body(toContent(io)).when().post("/").then().statusCode(SC_NOT_ACCEPTABLE);
}
}
}

View file

@ -28,6 +28,7 @@ import static org.mockito.Mockito.when;
import java.io.IOException;
import java.io.OutputStream;
import java.util.Collections;
import org.junit.Test;
import org.mockito.ArgumentCaptor;
@ -38,6 +39,7 @@ import com.sun.net.httpserver.HttpExchange;
import de.kosit.validationtool.api.Configuration;
import de.kosit.validationtool.config.TestConfigurationFactory;
import de.kosit.validationtool.impl.ConversionService;
import de.kosit.validationtool.impl.Helper;
/**
* @author Andreas Penski
@ -51,8 +53,8 @@ public class ConfigHandlerTest {
final OutputStream stream = mock(OutputStream.class);
when(exchange.getResponseHeaders()).thenReturn(headers);
when(exchange.getResponseBody()).thenReturn(stream);
final Configuration config = TestConfigurationFactory.createSimpleConfiguration().build();
final ConfigHandler handler = new ConfigHandler(config, new ConversionService());
final Configuration config = TestConfigurationFactory.createSimpleConfiguration().build(Helper.getTestProcessor());
final ConfigHandler handler = new ConfigHandler(Collections.singletonList(config), new ConversionService());
handler.handle(exchange);
verify(exchange, times(1)).sendResponseHeaders(HttpStatus.SC_OK, 0);
verify(stream, atLeast(1)).write(any());

View file

@ -21,7 +21,6 @@ import static org.assertj.core.api.Assertions.assertThat;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
@ -61,12 +60,6 @@ public class ContentRepositoryTest {
assertThat(schema).isNotNull();
}
@Test
public void testSchemaCaching() {
final Schema schema = this.repository.getReportInputSchema();
assertThat(this.repository.getReportInputSchema()).isSameAs(schema);
}
@Test
public void testCreateSchemaNotExisting() throws Exception {
this.exception.expect(IllegalStateException.class);
@ -112,36 +105,12 @@ public class ContentRepositoryTest {
@Test
public void loadFromJar() throws URISyntaxException {
assert Helper.JAR_REPOSITORY != null;
this.repository = new ContentRepository(ResolvingMode.STRICT_RELATIVE.getStrategy(), Helper.JAR_REPOSITORY.toURI());
this.repository = new ContentRepository(Helper.getTestProcessor(), ResolvingMode.STRICT_RELATIVE.getStrategy(),
Helper.JAR_REPOSITORY.toURI());
final XsltExecutable xsltExecutable = this.repository.loadXsltScript(URI.create("report.xsl"));
assertThat(xsltExecutable).isNotNull();
}
@Test
public void testLoadSchema() {
final URL main = RelativeUriResolverTest.class.getClassLoader().getResource("loading/main.xsd");
assert main != null;
final Schema schema = this.repository.createSchema(main, new ClassPathResourceResolver("/loading"));
final Schema schema = this.repository.createSchema(URI.create("main.xsd"));
assertThat(schema).isNotNull();
}
@Test
public void testLoadSchemaPackaged() throws URISyntaxException {
final URL main = RelativeUriResolverTest.class.getClassLoader().getResource("packaged/main.xsd");
assert main != null;
final Schema schema = this.repository.createSchema(main,
new ClassPathResourceResolver(RelativeUriResolverTest.class.getClassLoader().getResource("packaged/").toURI()));
assertThat(schema).isNotNull();
}
// @Test
// public void loadFromJar() throws URISyntaxException {
// this.content = new ContentRepository(TestObjectFactory.createProcessor(), Helper.JAR_REPOSITORY.toURI());
// this.repository = new ScenarioRepository(this.content);
// final CheckConfiguration conf = new CheckConfiguration(
// ScenarioRepository.class.getClassLoader().getResource("xrechnung/scenarios.xml").toURI());
// ScenarioRepository.initialize(conf);
// assertThat(this.repository.getScenarios()).isNotNull();
// }
}

View file

@ -65,16 +65,17 @@ public class DefaultCheckTest {
@Before
public void setup() throws URISyntaxException {
final Configuration validConfig = Configuration.load(Simple.SCENARIOS, Simple.REPOSITORY_URI).build();
final Configuration validConfig = Configuration.load(Simple.SCENARIOS, Simple.REPOSITORY_URI).build(Helper.getTestProcessor());
this.validCheck = new DefaultCheck(validConfig);
final Configuration errorConfig = Configuration.load(Simple.ERROR_SCENARIOS, Simple.REPOSITORY_URI).build();
final Configuration errorConfig = Configuration.load(Simple.ERROR_SCENARIOS, Simple.REPOSITORY_URI)
.build(Helper.getTestProcessor());
this.errorCheck = new DefaultCheck(errorConfig);
final Configuration jarConfig = Configuration
.load(requireNonNull(DefaultCheckTest.class.getClassLoader().getResource("simple/packaged/scenarios.xml")).toURI(),
requireNonNull(DefaultCheckTest.class.getClassLoader().getResource("simple/packaged/repository/")).toURI())
.build();
.build(Helper.getTestProcessor());
this.jarScenarioCheck = new DefaultCheck(jarConfig);
}
@ -248,8 +249,7 @@ public class DefaultCheckTest {
assertThat(result.isProcessingSuccessful()).isEqualTo(true);
// test compatible configuration
node = this.validCheck.getConfiguration().getContentRepository().getProcessor().newDocumentBuilder()
.build(new StreamSource(SIMPLE_VALID.toASCIIString()));
node = this.validCheck.getProcessor().newDocumentBuilder().build(new StreamSource(SIMPLE_VALID.toASCIIString()));
domInput = InputFactory.read(node, "node test");
result = this.validCheck.checkInput(domInput);
assertThat(result.isProcessingSuccessful()).isEqualTo(true);

View file

@ -30,6 +30,7 @@ import de.kosit.validationtool.api.Input;
import de.kosit.validationtool.api.ResolvingConfigurationStrategy;
import de.kosit.validationtool.impl.model.Result;
import de.kosit.validationtool.impl.tasks.DocumentParseAction;
import de.kosit.validationtool.impl.xml.ProcessorProvider;
import de.kosit.validationtool.model.reportInput.XMLSyntaxError;
import net.sf.saxon.s9api.Processor;
@ -61,6 +62,8 @@ public class Helper {
public static final URI SCENARIOS = ROOT.resolve("scenarios.xml");
public static final URI OTHER_SCENARIOS = ROOT.resolve("otherScenarios.xml");
public static final URI ERROR_SCENARIOS = ROOT.resolve("scenarios-with-errors.xml");
public static final URI REPOSITORY_URI = ROOT.resolve("repository/");
@ -83,7 +86,7 @@ public class Helper {
public static final ContentRepository createContentRepository() {
final ResolvingConfigurationStrategy strategy = ResolvingMode.STRICT_RELATIVE.getStrategy();
return new ContentRepository(strategy, Simple.REPOSITORY_URI);
return new ContentRepository(Helper.getTestProcessor(), strategy, Simple.REPOSITORY_URI);
}
public static URI getSchemaLocation() {
@ -122,6 +125,8 @@ public class Helper {
public static final URL JAR_REPOSITORY = Helper.class.getClassLoader().getResource("simple/packaged/repository/");
public static final URI LARGE_XML = Paths.get("pom.xml").toUri();
/**
* Lädt ein XML-Dokument von der gegebenen URL
*
@ -171,6 +176,6 @@ public class Helper {
}
public static Processor createProcessor() {
return ResolvingMode.STRICT_RELATIVE.getStrategy().getProcessor();
return ProcessorProvider.getProcessor();
}
}

View file

@ -55,7 +55,8 @@ public class ScenarioRepositoryTest {
@Before
public void setup() {
this.configInstance = new TestConfiguration();
this.configInstance.setContentRepository(new ContentRepository(ResolvingMode.STRICT_RELATIVE.getStrategy(), null));
this.configInstance
.setContentRepository(new ContentRepository(Helper.getTestProcessor(), ResolvingMode.STRICT_RELATIVE.getStrategy(), null));
final Scenario s = createScenario();
this.configInstance.setScenarios(new ArrayList<>());
@ -106,6 +107,25 @@ public class ScenarioRepositoryTest {
assertThat(scenario.getObject().getName()).isEqualTo("fallback");
}
@Test
public void testNoConfiguration() {
this.expectedException.expect(IllegalArgumentException.class);
this.repository = new ScenarioRepository();
}
@Test
public void testFallbackOnMultipleConfigurations() {
final TestConfiguration first = this.configInstance;
first.setFallbackScenario(createFallback());
setup();// create new one;
final TestConfiguration second = this.configInstance;
second.setFallbackScenario(createFallback());
this.repository = new ScenarioRepository(first, second);
final Scenario fallback = this.repository.getFallbackScenario();
assertThat(fallback).isSameAs(first.getFallbackScenario());
assertThat(fallback).isNotSameAs(second.getFallbackScenario());
}
private XdmNode load(final URI uri) throws IOException {
return Helper.parseDocument(this.configInstance.getContentRepository().getProcessor(), read(uri.toURL())).getObject();
}

View file

@ -40,7 +40,7 @@ public class SimpleScenarioCheckTest {
@Before
public void setup() {
final Configuration d = Configuration.load(Simple.SCENARIOS, Simple.REPOSITORY_URI).build();
final Configuration d = Configuration.load(Simple.SCENARIOS, Simple.REPOSITORY_URI).build(Helper.getTestProcessor());
this.implementation = new DefaultCheck(d);
}

View file

@ -16,8 +16,6 @@
package de.kosit.validationtool.impl;
import de.kosit.validationtool.impl.xml.StrictLocalResolvingStrategy;
import net.sf.saxon.s9api.Processor;
/**
@ -29,7 +27,7 @@ public class TestObjectFactory {
public static Processor createProcessor() {
if (processor == null) {
processor = new StrictLocalResolvingStrategy().getProcessor();
processor = Helper.getTestProcessor();
}
return processor;
}

View file

@ -59,25 +59,25 @@ public class VersioningTest {
@Test
public void testBase() throws URISyntaxException {
final Scenarios result = this.service.readXml(BASE.toURI(), Scenarios.class, this.repository.getScenarioSchema());
final Scenarios result = this.service.readXml(BASE.toURI(), Scenarios.class, SchemaProvider.getScenarioSchema());
assertThat(result).isNotNull();
}
@Test
public void testFrameworkIncrement() throws URISyntaxException {
final Scenarios result = this.service.readXml(INCREMENT.toURI(), Scenarios.class, this.repository.getScenarioSchema());
final Scenarios result = this.service.readXml(INCREMENT.toURI(), Scenarios.class, SchemaProvider.getScenarioSchema());
assertThat(result).isNotNull();
}
@Test
public void testNewFeature() throws URISyntaxException {
this.exception.expect(ConversionService.ConversionExeption.class);
this.service.readXml(NEW_FEATURE.toURI(), Scenarios.class, this.repository.getScenarioSchema());
this.service.readXml(NEW_FEATURE.toURI(), Scenarios.class, SchemaProvider.getScenarioSchema());
}
@Test
public void testNewVersion() throws URISyntaxException {
this.exception.expect(ConversionService.ConversionExeption.class);
this.service.readXml(NEW_VERSION.toURI(), Scenarios.class, this.repository.getScenarioSchema());
this.service.readXml(NEW_VERSION.toURI(), Scenarios.class, SchemaProvider.getScenarioSchema());
}
}

View file

@ -0,0 +1,61 @@
/*
* Copyright 2017-2020 Koordinierungsstelle für IT-Standards (KoSIT)
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package de.kosit.validationtool.impl.input;
import static org.assertj.core.api.Assertions.assertThat;
import java.io.BufferedInputStream;
import java.io.ByteArrayInputStream;
import java.io.FilterInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import org.apache.commons.io.IOUtils;
import org.junit.Test;
/**
* @author Andreas Penski
*/
public class StreamHelperTest {
/**
* Simulates a stream that is return 0 for {@link InputStream#available()} even though content is supplied.
*/
private static class MyLazyStream extends FilterInputStream {
protected MyLazyStream(final InputStream in) {
super(in);
}
@Override
public int available() throws IOException {
return 0;
}
}
@Test
public void testLazyStream() throws IOException {
final String myContent = "SomeBytes";
try ( final InputStream in = new MyLazyStream(new ByteArrayInputStream(myContent.getBytes())) ) {
final BufferedInputStream peekable = StreamHelper.wrapPeekable(in);
assertThat(peekable.available()).isGreaterThan(0);
final String read = IOUtils.toString(peekable, Charset.defaultCharset());
assertThat(read).isEqualTo(myContent);
}
}
}

View file

@ -26,6 +26,7 @@ import org.junit.Test;
import de.kosit.validationtool.api.AcceptRecommendation;
import de.kosit.validationtool.impl.ContentRepository;
import de.kosit.validationtool.impl.Helper;
import de.kosit.validationtool.impl.ResolvingMode;
import de.kosit.validationtool.impl.tasks.CheckAction.Bag;
@ -121,6 +122,7 @@ public class ComputeAcceptanceActionTest {
}
private static XPathExecutable createXpath(final String expression) {
return new ContentRepository(ResolvingMode.STRICT_RELATIVE.getStrategy(), null).createXPath(expression, new HashMap<>());
return new ContentRepository(Helper.getTestProcessor(), ResolvingMode.STRICT_RELATIVE.getStrategy(), null).createXPath(expression,
new HashMap<>());
}
}

View file

@ -54,8 +54,7 @@ public class CreateReportActionTest {
@Before
public void setup() {
this.repository = Simple.createContentRepository();
this.action = new CreateReportAction(this.repository.getProcessor(), new ConversionService(), this.repository.getResolver(),
this.repository.getUnparsedTextURIResolver());
this.action = new CreateReportAction(this.repository.getProcessor(), new ConversionService());
}
@Test
@ -85,7 +84,7 @@ public class CreateReportActionTest {
public void testExecutionException() throws SaxonApiException {
final Processor p = mock(Processor.class);
final DocumentBuilder documentBuilder = mock(DocumentBuilder.class);
this.action = new CreateReportAction(p, new ConversionService(), null, null);
this.action = new CreateReportAction(p, new ConversionService());
when(p.newDocumentBuilder()).thenReturn(documentBuilder);
when(documentBuilder.build(any(Source.class))).thenThrow(new SaxonApiException("mocked"));

View file

@ -44,10 +44,10 @@ import de.kosit.validationtool.api.XmlError.Severity;
import de.kosit.validationtool.impl.Helper;
import de.kosit.validationtool.impl.Helper.Simple;
import de.kosit.validationtool.impl.Scenario;
import de.kosit.validationtool.impl.SchemaProvider;
import de.kosit.validationtool.impl.TestObjectFactory;
import de.kosit.validationtool.impl.input.SourceInput;
import de.kosit.validationtool.impl.tasks.CheckAction.Bag;
import de.kosit.validationtool.impl.xml.StrictRelativeResolvingStrategy;
/**
* Tests die {@link SchemaValidationAction}.
@ -62,7 +62,7 @@ public class SchemaValidatorActionTest {
@Before
public void setup() {
this.service = new SchemaValidationAction(new StrictRelativeResolvingStrategy(), TestObjectFactory.createProcessor());
this.service = new SchemaValidationAction(TestObjectFactory.createProcessor());
}
@Test
@ -89,7 +89,7 @@ public class SchemaValidatorActionTest {
@Test
public void testSchemaReferences() {
final Schema reportInputSchema = Simple.createContentRepository().getReportInputSchema();
final Schema reportInputSchema = SchemaProvider.getReportInputSchema();
assertThat(reportInputSchema).isNotNull();
}

View file

@ -33,7 +33,6 @@ import de.kosit.validationtool.impl.ConversionService;
import de.kosit.validationtool.impl.Helper.Simple;
import de.kosit.validationtool.impl.Scenario;
import de.kosit.validationtool.impl.Scenario.Transformation;
import de.kosit.validationtool.impl.xml.RelativeUriResolver;
import de.kosit.validationtool.model.scenarios.ResourceType;
import net.sf.saxon.s9api.SaxonApiException;
@ -51,7 +50,7 @@ public class SchematronValidationActionTest {
@Before
public void setup() {
this.action = new SchematronValidationAction(new RelativeUriResolver(Simple.REPOSITORY_URI), new ConversionService());
this.action = new SchematronValidationAction(new ConversionService());
}
@Test

View file

@ -30,6 +30,7 @@ import org.oclc.purl.dsdl.svrl.SchematronOutput;
import de.kosit.validationtool.api.Input;
import de.kosit.validationtool.api.InputFactory;
import de.kosit.validationtool.api.ResolvingConfigurationStrategy;
import de.kosit.validationtool.impl.ContentRepository;
import de.kosit.validationtool.impl.Helper;
import de.kosit.validationtool.impl.ResolvingMode;
@ -82,6 +83,8 @@ public class TestBagBuilder {
t.setValidateWithXmlSchema(v);
final Scenario scenario = new Scenario(t);
scenario.setSchema(createSchema(schemafile.toURL()));
final ResolvingConfigurationStrategy strategy = ResolvingMode.STRICT_RELATIVE.getStrategy();
scenario.setFactory(strategy);
return scenario;
} catch (final MalformedURLException e) {
throw new IllegalArgumentException(e);
@ -89,7 +92,8 @@ public class TestBagBuilder {
}
private static Schema createSchema(final URL toURL) {
final ContentRepository contentRepository = new ContentRepository(ResolvingMode.STRICT_RELATIVE.getStrategy(), null);
final ContentRepository contentRepository = new ContentRepository(Helper.getTestProcessor(),
ResolvingMode.STRICT_RELATIVE.getStrategy(), null);
return contentRepository.createSchema(toURL);
}

View file

@ -0,0 +1,30 @@
/*
* Copyright 2017-2020 Koordinierungsstelle für IT-Standards (KoSIT)
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package de.kosit.validationtool.impl.xml;
import javax.xml.validation.SchemaFactory;
import de.kosit.validationtool.impl.ResolvingMode;
/**
* @author Andreas Penski
*/
public class SchemaProviderTest {
private final SchemaFactory schemaFactory = ResolvingMode.STRICT_RELATIVE.getStrategy().createSchemaFactory();
}

View file

@ -0,0 +1,95 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Copyright 2017-2020 Koordinierungsstelle für IT-Standards (KoSIT)
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<scenarios xmlns="http://www.xoev.de/de/validator/framework/1/scenarios" frameworkVersion="1.1.2">
<name>HTML-TestSuite</name>
<author>QA</author>
<date>2017-08-08</date>
<description>
<p>Szenario für Tests</p>
</description>
<scenario>
<name>Simple</name>
<description>
<p>Nur Schemaprüfung.</p>
</description>
<namespace prefix="cri">http://www.xoev.de/de/validator/framework/1/createreportinput
</namespace>
<namespace prefix="test">
http://validator.kosit.de/test-sample</namespace>
<namespace prefix="rpt">http://validator.kosit.de/test-report</namespace>
<match>/test:doesNotMatch</match>
<validateWithXmlSchema>
<resource>
<name>Sample Schema</name>
<location>simple.xsd</location>
</resource>
</validateWithXmlSchema>
<validateWithSchematron>
<resource>
<name>Sample Schematron</name>
<location>simple.xsl</location>
</resource>
</validateWithSchematron>
<createReport>
<resource>
<name>Report für eRechnung</name>
<location>report.xsl</location>
</resource>
</createReport>
<acceptMatch>count(//test:rejected) = 0</acceptMatch>
</scenario>
<scenario>
<name>NoAcceptMatch</name>
<description>
<p>Nur Schemaprüfung. Keine AcceptMatch deklaration</p>
<p>Testen, ob auch alte Konfiguration funktionioeren</p>
</description>
<namespace prefix="test">http://validator.kosit.de/test-sample</namespace>
<match>/test:DoesNotMatchfoo</match>
<validateWithXmlSchema>
<resource>
<name>Sample Schema</name>
<location>simple.xsd</location>
</resource>
</validateWithXmlSchema>
<validateWithSchematron>
<resource>
<name>Sample Schematron</name>
<location>simple.xsl</location>
</resource>
</validateWithSchematron>
<createReport>
<resource>
<name>Report für eRechnung</name>
<location>report.xsl</location>
</resource>
</createReport>
</scenario>
<noScenarioReport>
<resource>
<name>default</name>
<location>report.xsl</location>
</resource>
</noScenarioReport>
</scenarios>