Browse Source

Add K/Wasm D8 target to the benchmarks (#5277)

- Updated the README.md with the description of benchmarks (might be
useful to external parties)
- customized a runner for D8
- added a `warmupCount` parameter to `runBenchmarks`, because it's
required for Jetstream3 (a set of benchmarks) that there is no warmup
(they skip it)
- Added a smooth scroll variant of LazyGrid + variants with
LaunchedEffect in grid cells for coroutines usage.
- Added a new class `Config`. Object `Args` is simplified, it's purpose
is to simply parse the arguments and produce a instance of `Config`. The
usages of `Args` were replaced by `Config`, except the parsing in fun
main.
- Renamed Example1 to MultipleComponents
- Also added MultipleComponents-NoVectorGraphics for D8 - it's similar
to MultipleComponents but doesn't use vector icons.

Fixes https://youtrack.jetbrains.com/issue/CMP-6942

Note: it's based on this PR
https://github.com/JetBrains/compose-multiplatform/pull/5275

## Testing
Manually run the benchmarks

## Release Notes
N/A
pull/5302/head
Oleksandr Karpovich 8 months ago committed by GitHub
parent
commit
8fac609b6c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
  1. 31
      benchmarks/multiplatform/README.md
  2. 50
      benchmarks/multiplatform/benchmarks/build.gradle.kts
  3. BIN
      benchmarks/multiplatform/benchmarks/src/commonMain/composeResources/drawable/compose-multiplatform.png
  4. 36
      benchmarks/multiplatform/benchmarks/src/commonMain/composeResources/drawable/compose-multiplatform.xml
  5. 123
      benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/Args.kt
  6. 60
      benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/Benchmarks.kt
  7. 4
      benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/BenchmarksSave.kt
  8. 25
      benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/MeasureComposable.kt
  9. 63
      benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/benchmarks/lazygrid/LazyGrid.kt
  10. 2
      benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/benchmarks/multipleComponents/Clickable.kt
  11. 34
      benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/benchmarks/multipleComponents/MultipleComponents.kt
  12. 2
      benchmarks/multiplatform/benchmarks/src/desktopMain/kotlin/main.desktop.kt
  13. 2
      benchmarks/multiplatform/benchmarks/src/iosMain/kotlin/main.ios.kt
  14. 2
      benchmarks/multiplatform/benchmarks/src/macosMain/kotlin/main.macos.kt
  15. 74
      benchmarks/multiplatform/benchmarks/src/wasmJsMain/kotlin/main.wasmJs.kt
  16. 13
      benchmarks/multiplatform/benchmarks/src/wasmJsMain/resources/launcher.mjs
  17. 24
      benchmarks/multiplatform/benchmarks/src/wasmJsMain/resources/launcher_jetstream3.mjs
  18. 56
      benchmarks/multiplatform/benchmarks/src/wasmJsMain/resources/polyfills.mjs

31
benchmarks/multiplatform/README.md

@ -12,10 +12,39 @@ Alternatively you may open `iosApp/iosApp` project in XCode and run the app from @@ -12,10 +12,39 @@ Alternatively you may open `iosApp/iosApp` project in XCode and run the app from
- `./gradlew :benchmarks:runReleaseExecutableMacosArm64` (Works on Arm64 processors)
- `./gradlew :benchmarks:runReleaseExecutableMacosX64` (Works on Intel processors)
## Run K/Wasm target in D8:
`./gradlew :benchmarks:wasmJsD8ProductionRun`
or with arguments:
`./gradlew :benchmarks:wasmJsD8ProductionRun -PrunArguments=benchmarks=AnimatedVisibility`
## To build and run a K/Wasm D8 distribution for Jetstream3-like:
`./gradlew :benchmarks:buildD8Distribution --rerun-tasks`
then in a distribution directory run using your D8 binary:
`~/.gradle/d8/v8-mac-arm64-rel-11.9.85/d8 --module launcher_jetstream3.mjs -- AnimatedVisibility 1000`
## Run in web browser:
Please run your browser with manual GC enabled before running the benchmark, like for Google Chrome:
`open -a Google\ Chrome --args --js-flags="--expose-gc"`
- `./gradlew :benchmarks:wasmJsBrowserProductionRun` (you can see the results printed on the page itself)
- `./gradlew clean :benchmarks:wasmJsBrowserProductionRun` (you can see the results printed on the page itself)
# Benchmarks description
| Benchmark Name | File Path | Description |
|------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------|
| AnimatedVisibility | [benchmarks/src/commonMain/kotlin/benchmarks/animation/AnimatedVisibility.kt](benchmarks/src/commonMain/kotlin/benchmarks/animation/AnimatedVisibility.kt) | Tests the performance of the AnimatedVisibility component by repeatedly toggling the visibility of a PNG image. |
| LazyGrid | [benchmarks/src/commonMain/kotlin/benchmarks/lazygrid/LazyGrid.kt](benchmarks/src/commonMain/kotlin/benchmarks/lazygrid/LazyGrid.kt) | Tests the performance of the LazyVerticalGrid component with 12,000 items and jumps to specific items multiple times while running. |
| LazyGrid-ItemLaunchedEffect | [benchmarks/src/commonMain/kotlin/benchmarks/lazygrid/LazyGrid.kt](benchmarks/src/commonMain/kotlin/benchmarks/lazygrid/LazyGrid.kt) | Same as LazyGrid but adds a LaunchedEffect in each grid item that simulates an async task. |
| LazyGrid-SmoothScroll | [benchmarks/src/commonMain/kotlin/benchmarks/lazygrid/LazyGrid.kt](benchmarks/src/commonMain/kotlin/benchmarks/lazygrid/LazyGrid.kt) | Same as LazyGrid but uses smooth scrolling instead of jumping to items. |
| LazyGrid-SmoothScroll-ItemLaunchedEffect | [benchmarks/src/commonMain/kotlin/benchmarks/lazygrid/LazyGrid.kt](benchmarks/src/commonMain/kotlin/benchmarks/lazygrid/LazyGrid.kt) | Combines smooth scrolling with LaunchedEffect in each item. |
| VisualEffects | [benchmarks/src/commonMain/kotlin/benchmarks/visualeffects/HappyNY.kt](benchmarks/src/commonMain/kotlin/benchmarks/visualeffects/HappyNY.kt) | Tests the performance of complex animations and visual effects including snow flakes, stars, and rocket particles. |
| LazyList | [benchmarks/src/commonMain/kotlin/benchmarks/complexlazylist/components/MainUI.kt](benchmarks/src/commonMain/kotlin/benchmarks/complexlazylist/components/MainUI.kt) | Tests the performance of a complex LazyColumn implementation with features like pull-to-refresh, loading more items, and continuous scrolling. |
| MultipleComponents | [benchmarks/src/commonMain/kotlin/benchmarks/example1/Example1.kt](benchmarks/src/commonMain/kotlin/benchmarks/multipleComponents/MultipleComponents.kt) | Tests the performance of a comprehensive UI that showcases various Compose components including layouts, animations, and styled text. |
| MultipleComponents-NoVectorGraphics | [benchmarks/src/commonMain/kotlin/benchmarks/example1/Example1.kt](benchmarks/src/commonMain/kotlin/benchmarks/multipleComponents/MultipleComponents.kt) | Same as MultipleComponents but skips the Composables with vector graphics rendering. |

50
benchmarks/multiplatform/benchmarks/build.gradle.kts

@ -1,4 +1,6 @@ @@ -1,4 +1,6 @@
import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl
import org.jetbrains.kotlin.gradle.targets.js.binaryen.BinaryenRootEnvSpec
import org.jetbrains.kotlin.gradle.targets.js.d8.D8Exec
import org.jetbrains.kotlin.gradle.targets.js.webpack.KotlinWebpack
import kotlin.text.replace
@ -46,7 +48,14 @@ kotlin { @@ -46,7 +48,14 @@ kotlin {
@OptIn(ExperimentalWasmDsl::class)
wasmJs {
binaries.executable()
browser ()
d8 {
compilerOptions.freeCompilerArgs.add("-Xwasm-attach-js-exception")
runTask {
// It aborts even on coroutine cancellation exceptions:
// d8Args.add("--abort-on-uncaught-exception")
}
}
browser()
}
sourceSets {
@ -114,4 +123,43 @@ gradle.taskGraph.whenReady { @@ -114,4 +123,43 @@ gradle.taskGraph.whenReady {
open = "http://localhost:8080?$args"
)
}
@OptIn(ExperimentalWasmDsl::class)
tasks.withType<D8Exec>().configureEach {
inputFileProperty.set(rootProject.layout.buildDirectory.file(
"js/packages/compose-benchmarks-benchmarks-wasm-js/kotlin/launcher.mjs")
)
args(appArgs)
}
}
tasks.register("buildD8Distribution", Zip::class.java) {
dependsOn("wasmJsProductionExecutableCompileSync")
from(rootProject.layout.buildDirectory.file("js/packages/compose-benchmarks-benchmarks-wasm-js/kotlin"))
archiveFileName.set("d8-distribution.zip")
destinationDirectory.set(rootProject.layout.buildDirectory.dir("distributions"))
}
tasks.withType<org.jetbrains.kotlin.gradle.targets.js.binaryen.BinaryenExec>().configureEach {
binaryenArgs.add("-g") // keep the readable names
}
@OptIn(ExperimentalWasmDsl::class)
rootProject.the<BinaryenRootEnvSpec>().apply {
// version = "122" // change only if needed
}
val jsOrWasmRegex = Regex("js|wasm")
configurations.all {
resolutionStrategy.eachDependency {
if (requested.group.startsWith("org.jetbrains.skiko") &&
jsOrWasmRegex.containsMatchIn(requested.name)
) {
// to keep the readable names from Skiko
useVersion(requested.version!! + "+profiling")
}
}
}

BIN
benchmarks/multiplatform/benchmarks/src/commonMain/composeResources/drawable/compose-multiplatform.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

36
benchmarks/multiplatform/benchmarks/src/commonMain/composeResources/drawable/compose-multiplatform.xml

@ -1,36 +0,0 @@ @@ -1,36 +0,0 @@
<vector xmlns:android="http://schemas.android.com/apk/res/android"
android:width="600dp"
android:height="600dp"
android:viewportWidth="600"
android:viewportHeight="600">
<path
android:pathData="M301.21,418.53C300.97,418.54 300.73,418.56 300.49,418.56C297.09,418.59 293.74,417.72 290.79,416.05L222.6,377.54C220.63,376.43 219,374.82 217.85,372.88C216.7,370.94 216.09,368.73 216.07,366.47L216.07,288.16C216.06,287.32 216.09,286.49 216.17,285.67C216.38,283.54 216.91,281.5 217.71,279.6L199.29,268.27L177.74,256.19C175.72,260.43 174.73,265.23 174.78,270.22L174.79,387.05C174.85,393.89 178.57,400.2 184.53,403.56L286.26,461.02C290.67,463.51 295.66,464.8 300.73,464.76C300.91,464.76 301.09,464.74 301.27,464.74C301.24,449.84 301.22,439.23 301.22,439.23L301.21,418.53Z"
android:fillColor="#041619"
android:fillType="nonZero"/>
<path
android:pathData="M409.45,242.91L312.64,188.23C303.64,183.15 292.58,183.26 283.68,188.51L187.92,245C183.31,247.73 179.93,251.62 177.75,256.17L177.74,256.19L199.29,268.27L217.71,279.6C217.83,279.32 217.92,279.02 218.05,278.74C218.24,278.36 218.43,277.98 218.64,277.62C219.06,276.88 219.52,276.18 220.04,275.51C221.37,273.8 223.01,272.35 224.87,271.25L289.06,233.39C290.42,232.59 291.87,231.96 293.39,231.51C295.53,230.87 297.77,230.6 300,230.72C302.98,230.88 305.88,231.73 308.47,233.2L373.37,269.85C375.54,271.08 377.49,272.68 379.13,274.57C379.68,275.19 380.18,275.85 380.65,276.53C380.86,276.84 381.05,277.15 381.24,277.47L397.79,266.39L420.34,252.93L420.31,252.88C417.55,248.8 413.77,245.35 409.45,242.91Z"
android:fillColor="#37BF6E"
android:fillType="nonZero"/>
<path
android:pathData="M381.24,277.47C381.51,277.92 381.77,278.38 382.01,278.84C382.21,279.24 382.39,279.65 382.57,280.06C382.91,280.88 383.19,281.73 383.41,282.59C383.74,283.88 383.92,285.21 383.93,286.57L383.93,361.1C383.96,363.95 383.35,366.77 382.16,369.36C381.93,369.86 381.69,370.35 381.42,370.83C379.75,373.79 377.32,376.27 374.39,378L310.2,415.87C307.47,417.48 304.38,418.39 301.21,418.53L301.22,439.23C301.22,439.23 301.24,449.84 301.27,464.74C306.1,464.61 310.91,463.3 315.21,460.75L410.98,404.25C419.88,399 425.31,389.37 425.22,379.03L425.22,267.85C425.17,262.48 423.34,257.34 420.34,252.93L397.79,266.39L381.24,277.47Z"
android:fillColor="#3870B2"
android:fillType="nonZero"/>
<path
android:pathData="M177.75,256.17C179.93,251.62 183.31,247.73 187.92,245L283.68,188.51C292.58,183.26 303.64,183.15 312.64,188.23L409.45,242.91C413.77,245.35 417.55,248.8 420.31,252.88L420.34,252.93L498.59,206.19C494.03,199.46 487.79,193.78 480.67,189.75L320.86,99.49C306.01,91.1 287.75,91.27 273.07,99.95L114.99,193.2C107.39,197.69 101.81,204.11 98.21,211.63L177.74,256.19L177.75,256.17ZM301.27,464.74C301.09,464.74 300.91,464.76 300.73,464.76C295.66,464.8 290.67,463.51 286.26,461.02L184.53,403.56C178.57,400.2 174.85,393.89 174.79,387.05L174.78,270.22C174.73,265.23 175.72,260.43 177.74,256.19L98.21,211.63C94.86,218.63 93.23,226.58 93.31,234.82L93.31,427.67C93.42,438.97 99.54,449.37 109.4,454.92L277.31,549.77C284.6,553.88 292.84,556.01 301.2,555.94L301.2,555.8C301.39,543.78 301.33,495.26 301.27,464.74Z"
android:strokeWidth="10"
android:fillColor="#00000000"
android:strokeColor="#083042"
android:fillType="nonZero"/>
<path
android:pathData="M498.59,206.19L420.34,252.93C423.34,257.34 425.17,262.48 425.22,267.85L425.22,379.03C425.31,389.37 419.88,399 410.98,404.25L315.21,460.75C310.91,463.3 306.1,464.61 301.27,464.74C301.33,495.26 301.39,543.78 301.2,555.8L301.2,555.94C309.48,555.87 317.74,553.68 325.11,549.32L483.18,456.06C497.87,447.39 506.85,431.49 506.69,414.43L506.69,230.91C506.6,222.02 503.57,213.5 498.59,206.19Z"
android:strokeWidth="10"
android:fillColor="#00000000"
android:strokeColor="#083042"
android:fillType="nonZero"/>
<path
android:pathData="M301.2,555.94C292.84,556.01 284.6,553.88 277.31,549.76L109.4,454.92C99.54,449.37 93.42,438.97 93.31,427.67L93.31,234.82C93.23,226.58 94.86,218.63 98.21,211.63C101.81,204.11 107.39,197.69 114.99,193.2L273.07,99.95C287.75,91.27 306.01,91.1 320.86,99.49L480.67,189.75C487.79,193.78 494.03,199.46 498.59,206.19C503.57,213.5 506.6,222.02 506.69,230.91L506.69,414.43C506.85,431.49 497.87,447.39 483.18,456.06L325.11,549.32C317.74,553.68 309.48,555.87 301.2,555.94Z"
android:strokeWidth="10"
android:fillColor="#00000000"
android:strokeColor="#083042"
android:fillType="nonZero"/>
</vector>

123
benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/Args.kt

@ -4,18 +4,6 @@ enum class Mode { @@ -4,18 +4,6 @@ enum class Mode {
}
object Args {
private val modes = mutableSetOf<Mode>()
private val benchmarks = mutableMapOf<String, Int>()
var versionInfo: String? = null
private set
var saveStatsToCSV: Boolean = false
private set
var saveStatsToJSON: Boolean = false
private set
private fun argToSet(arg: String): Set<String> = arg.substring(arg.indexOf('=') + 1)
.split(",").filter{!it.isEmpty()}.map{it.uppercase()}.toSet()
@ -30,14 +18,26 @@ object Args { @@ -30,14 +18,26 @@ object Args {
}
}
private fun String.decodeArg() = replace("%20", " ")
/**
* Parses command line arguments to determine modes and benchmarks settings.
* Parses command line arguments to create a [Config] for benchmarks run.
*
* @param args an array of strings representing the command line arguments.
* Each argument can specify either "modes" or "benchmarks" settings,
* with values separated by commas.
* Each argument can specify either of these settings:
* modes, benchmarks, disabledBenchmarks - comma separated values,
* versionInfo, saveStatsToCSV, saveStatsToJSON - single values.
*
* Example: benchmarks=AnimatedVisibility(100),modes=SIMPLE,versionInfo=Kotlin_2_1_20,saveStatsToCSV=true
*/
fun parseArgs(args: Array<String>) {
fun parseArgs(args: Array<String>): Config {
val modes = mutableSetOf<Mode>()
val benchmarks = mutableMapOf<String, Int>()
val disabledBenchmarks = mutableSetOf<String>()
var versionInfo: String? = null
var saveStatsToCSV: Boolean = false
var saveStatsToJSON: Boolean = false
for (arg in args) {
if (arg.startsWith("modes=", ignoreCase = true)) {
modes.addAll(argToSet(arg.decodeArg()).map { Mode.valueOf(it) })
@ -49,18 +49,99 @@ object Args { @@ -49,18 +49,99 @@ object Args {
saveStatsToCSV = arg.substringAfter("=").toBoolean()
} else if (arg.startsWith("saveStatsToJSON=", ignoreCase = true)) {
saveStatsToJSON = arg.substringAfter("=").toBoolean()
} else if (arg.startsWith("disabledBenchmarks=", ignoreCase = true)) {
disabledBenchmarks += argToMap(arg.decodeArg()).keys
}
}
}
private fun String.decodeArg() = replace("%20", " ")
return Config(
modes = modes,
benchmarks = benchmarks,
disabledBenchmarks = disabledBenchmarks,
versionInfo = versionInfo,
saveStatsToCSV = saveStatsToCSV,
saveStatsToJSON = saveStatsToJSON
)
}
}
/**
* Represents the benchmarks configuration parsed from command line arguments or configured programmatically.
*
* @property modes The set of enabled execution modes. If empty, all modes are considered enabled by default checks.
* @property benchmarks A map of explicitly mentioned benchmarks to their specific problem sizes.
* A value of -1 indicates the benchmark is enabled but should use its default size.
* If the map is empty, all benchmarks are considered enabled by default checks.
* @property disabledBenchmarks A set of benchmarks to skip.
* @property versionInfo Optional string containing version information.
* @property saveStatsToCSV Flag indicating whether statistics should be saved to a CSV file.
* @property saveStatsToJSON Flag indicating whether statistics should be saved to a JSON file.
*/
data class Config(
val modes: Set<Mode> = emptySet(),
val benchmarks: Map<String, Int> = emptyMap(), // Name -> Problem Size (-1 for default)
val disabledBenchmarks: Set<String> = emptySet(),
val versionInfo: String? = null,
val saveStatsToCSV: Boolean = false,
val saveStatsToJSON: Boolean = false
) {
/**
* Checks if a specific mode is enabled based on the configuration.
* A mode is considered enabled if no modes were specified (default) or if it's explicitly listed.
*/
fun isModeEnabled(mode: Mode): Boolean = modes.isEmpty() || modes.contains(mode)
fun isBenchmarkEnabled(benchmark: String): Boolean = benchmarks.isEmpty() || benchmarks.contains(benchmark.uppercase())
/**
* Checks if a specific benchmark is enabled
*/
fun isBenchmarkEnabled(benchmark: String): Boolean {
val normalizedName = benchmark.uppercase()
// Enabled if the benchmarks map is empty OR if the specific benchmark is present
return (benchmarks.isEmpty() || benchmarks.containsKey(normalizedName))
&& !disabledBenchmarks.contains(normalizedName)
&& !disabledBenchmarks.contains(benchmark)
}
/**
* Returns the problem size configured for [benchmark], or [default] if not set.
*
* @param benchmark Benchmark name (case-insensitive).
* @param default Fallback size when no configuration is found.
* @return The problem size to use.
*/
fun getBenchmarkProblemSize(benchmark: String, default: Int): Int {
val result = benchmarks[benchmark.uppercase()]?: -1
return if (result == -1) default else result
val normalizedName = benchmark.uppercase()
val problemSize = benchmarks[normalizedName] ?: -1
return if (problemSize == -1) default else problemSize
}
companion object {
private var global: Config = Config()
val versionInfo: String?
get() = global.versionInfo
val saveStatsToCSV: Boolean
get() = global.saveStatsToCSV
val saveStatsToJSON: Boolean
get() = global.saveStatsToJSON
fun setGlobal(global: Config) {
this.global = global
}
fun setGlobalFromArgs(args: Array<String>) {
this.global = Args.parseArgs(args)
}
fun isModeEnabled(mode: Mode): Boolean =
global.isModeEnabled(mode)
fun isBenchmarkEnabled(benchmark: String): Boolean =
global.isBenchmarkEnabled(benchmark)
fun getBenchmarkProblemSize(benchmark: String, default: Int): Int =
global.getBenchmarkProblemSize(benchmark, default)
}
}

60
benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/Benchmarks.kt

@ -1,10 +1,11 @@ @@ -1,10 +1,11 @@
import androidx.compose.runtime.Composable
import benchmarks.animation.AnimatedVisibility
import benchmarks.complexlazylist.components.MainUiNoImageUseModel
import benchmarks.example1.Example1
import benchmarks.multipleComponents.MultipleComponentsExample
import benchmarks.lazygrid.LazyGrid
import benchmarks.visualeffects.NYContent
import kotlinx.serialization.Serializable
import kotlinx.serialization.Transient
import kotlinx.serialization.json.Json
import kotlin.math.roundToInt
import kotlin.time.Duration
@ -103,21 +104,21 @@ data class BenchmarkStats( @@ -103,21 +104,21 @@ data class BenchmarkStats(
val percentileCPUAverage: List<BenchmarkPercentileAverage>,
val percentileGPUAverage: List<BenchmarkPercentileAverage>,
val noBufferingMissedFrames: MissedFrames,
val doubleBufferingMissedFrames: MissedFrames
val doubleBufferingMissedFrames: MissedFrames,
) {
fun prettyPrint() {
val versionInfo = Args.versionInfo
val versionInfo = Config.versionInfo
if (versionInfo != null) {
println("Version: $versionInfo")
}
conditions.prettyPrint()
println()
if (Args.isModeEnabled(Mode.SIMPLE)) {
if (Config.isModeEnabled(Mode.SIMPLE)) {
val frameInfo = requireNotNull(averageFrameInfo) { "frameInfo shouldn't be null with Mode.SIMPLE" }
frameInfo.prettyPrint()
println()
}
if (Args.isModeEnabled(Mode.VSYNC_EMULATION)) {
if (Config.isModeEnabled(Mode.VSYNC_EMULATION)) {
percentileCPUAverage.prettyPrint(BenchmarkFrameTimeKind.CPU)
println()
percentileGPUAverage.prettyPrint(BenchmarkFrameTimeKind.GPU)
@ -128,16 +129,16 @@ data class BenchmarkStats( @@ -128,16 +129,16 @@ data class BenchmarkStats(
}
fun putFormattedValuesTo(map: MutableMap<String, String>) {
val versionInfo = Args.versionInfo
val versionInfo = Config.versionInfo
if (versionInfo != null) {
map.put("Version", versionInfo)
}
conditions.putFormattedValuesTo(map)
if (Args.isModeEnabled(Mode.SIMPLE)) {
if (Config.isModeEnabled(Mode.SIMPLE)) {
val frameInfo = requireNotNull(averageFrameInfo) { "frameInfo shouldn't be null with Mode.SIMPLE" }
frameInfo.putFormattedValuesTo(map)
}
if (Args.isModeEnabled(Mode.VSYNC_EMULATION)) {
if (Config.isModeEnabled(Mode.VSYNC_EMULATION)) {
percentileCPUAverage.putFormattedValuesTo(BenchmarkFrameTimeKind.CPU, map)
percentileGPUAverage.putFormattedValuesTo(BenchmarkFrameTimeKind.GPU, map)
noBufferingMissedFrames.putFormattedValuesTo("no buffering", map)
@ -233,20 +234,20 @@ suspend fun runBenchmark( @@ -233,20 +234,20 @@ suspend fun runBenchmark(
warmupCount: Int = 100,
content: @Composable () -> Unit
) {
if (Args.isBenchmarkEnabled(name)) {
if (Config.isBenchmarkEnabled(name)) {
println("# $name")
val stats = measureComposable(
name,
warmupCount,
Args.getBenchmarkProblemSize(name, frameCount),
width,
height,
targetFps,
graphicsContext,
content
name = name,
warmupCount = warmupCount,
frameCount = Config.getBenchmarkProblemSize(name, frameCount),
width = width,
height = height,
targetFps = targetFps,
graphicsContext = graphicsContext,
content = content
).generateStats()
stats.prettyPrint()
saveBenchmarkStatsOnDisk(name, stats)
saveBenchmarkStatsOnDisk(name = name, stats = stats)
}
}
@ -254,14 +255,27 @@ suspend fun runBenchmarks( @@ -254,14 +255,27 @@ suspend fun runBenchmarks(
width: Int = 1920,
height: Int = 1080,
targetFps: Int = 120,
warmupCount: Int = 100,
graphicsContext: GraphicsContext? = null
) {
println()
println("Running emulating $targetFps FPS")
println()
runBenchmark("AnimatedVisibility", width, height, targetFps, 1000, graphicsContext) { AnimatedVisibility() }
runBenchmark("LazyGrid", width, height, targetFps, 1000, graphicsContext) { LazyGrid() }
runBenchmark("VisualEffects", width, height, targetFps, 1000, graphicsContext) { NYContent(width, height) }
runBenchmark("LazyList", width, height, targetFps, 1000, graphicsContext) { MainUiNoImageUseModel() }
runBenchmark("Example1", width, height, targetFps, 1000, graphicsContext) { Example1() }
runBenchmark("AnimatedVisibility", width, height, targetFps, 1000, graphicsContext, warmupCount) { AnimatedVisibility() }
runBenchmark("LazyGrid", width, height, targetFps, 1000, graphicsContext, warmupCount) { LazyGrid() }
runBenchmark("LazyGrid-ItemLaunchedEffect", width, height, targetFps, 1000, graphicsContext, warmupCount) {
LazyGrid(smoothScroll = false, withLaunchedEffectInItem = true)
}
runBenchmark("LazyGrid-SmoothScroll", width, height, targetFps, 1000, graphicsContext, warmupCount) {
LazyGrid(smoothScroll = true)
}
runBenchmark("LazyGrid-SmoothScroll-ItemLaunchedEffect", width, height, targetFps, 1000, graphicsContext, warmupCount) {
LazyGrid(smoothScroll = true, withLaunchedEffectInItem = true)
}
runBenchmark("VisualEffects", width, height, targetFps, 1000, graphicsContext, warmupCount) { NYContent(width, height) }
runBenchmark("LazyList", width, height, targetFps, 1000, graphicsContext, warmupCount) { MainUiNoImageUseModel()}
runBenchmark("MultipleComponents", width, height, targetFps, 1000, graphicsContext, warmupCount) { MultipleComponentsExample() }
runBenchmark("MultipleComponents-NoVectorGraphics", width, height, targetFps, 1000, graphicsContext, warmupCount) {
MultipleComponentsExample(isVectorGraphicsSupported = false)
}
}

4
benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/BenchmarksSave.kt

@ -19,7 +19,7 @@ import kotlinx.io.readByteArray @@ -19,7 +19,7 @@ import kotlinx.io.readByteArray
fun saveBenchmarkStatsOnDisk(name: String, stats: BenchmarkStats) {
try {
if (Args.saveStatsToCSV) {
if (Config.saveStatsToCSV) {
val path = Path("build/benchmarks/$name.csv")
val keyToValue = mutableMapOf<String, String>()
@ -39,7 +39,7 @@ fun saveBenchmarkStatsOnDisk(name: String, stats: BenchmarkStats) { @@ -39,7 +39,7 @@ fun saveBenchmarkStatsOnDisk(name: String, stats: BenchmarkStats) {
SystemFileSystem.sink(path).writeText(text)
println("CSV results saved to ${SystemFileSystem.resolve(path)}")
println()
} else if (Args.saveStatsToJSON) {
} else if (Config.saveStatsToJSON) {
val jsonString = stats.toJsonString()
val jsonPath = Path("build/benchmarks/json-reports/$name.json")

25
benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/MeasureComposable.kt

@ -40,8 +40,20 @@ suspend inline fun preciseDelay(duration: Duration) { @@ -40,8 +40,20 @@ suspend inline fun preciseDelay(duration: Duration) {
while (liveDelayStart.elapsedNow() < liveDelay){}
}
/**
* Some of the benchmarks involved an asynchronous fetch operation for resources when running in a browser.
* To let the fetch operation result be handled by compose, the benchmark loop must yield the event loop.
* Otherwise, such benchmark do not run a part of workload, making the stats less meaningful.
*
* It makes sense for all platforms, since there could be some work scheduled to run on the same Thread
* as the benchmarks runner. But without yielding, it won't be dispatched.
*/
private suspend inline fun yieldEventLoop() {
yield()
}
@OptIn(ExperimentalTime::class, InternalComposeUiApi::class)
suspend fun measureComposable(
internal suspend fun measureComposable(
name: String,
warmupCount: Int,
frameCount: Int,
@ -62,13 +74,14 @@ suspend fun measureComposable( @@ -62,13 +74,14 @@ suspend fun measureComposable(
scene.mimicSkikoRender(surface, it * nanosPerFrame, width, height)
surface.flushAndSubmit(false)
graphicsContext?.awaitGPUCompletion()
yieldEventLoop()
}
runGC()
var cpuTotalTime = Duration.ZERO
var gpuTotalTime = Duration.ZERO
if (Args.isModeEnabled(Mode.SIMPLE)) {
if (Config.isModeEnabled(Mode.SIMPLE)) {
cpuTotalTime = measureTime {
repeat(frameCount) {
scene.mimicSkikoRender(surface, it * nanosPerFrame, width, height)
@ -76,6 +89,7 @@ suspend fun measureComposable( @@ -76,6 +89,7 @@ suspend fun measureComposable(
gpuTotalTime += measureTime {
graphicsContext?.awaitGPUCompletion()
}
yieldEventLoop()
}
}
cpuTotalTime -= gpuTotalTime
@ -85,8 +99,7 @@ suspend fun measureComposable( @@ -85,8 +99,7 @@ suspend fun measureComposable(
BenchmarkFrame(Duration.INFINITE, Duration.INFINITE)
}
if (Args.isModeEnabled(Mode.VSYNC_EMULATION)) {
if (Config.isModeEnabled(Mode.VSYNC_EMULATION)) {
var nextVSync = Duration.ZERO
var missedFrames = 0;
@ -118,6 +131,8 @@ suspend fun measureComposable( @@ -118,6 +131,8 @@ suspend fun measureComposable(
// Emulate waiting for next vsync
preciseDelay(timeUntilNextVSync)
}
yieldEventLoop()
}
}
@ -126,7 +141,7 @@ suspend fun measureComposable( @@ -126,7 +141,7 @@ suspend fun measureComposable(
nanosPerFrame.nanoseconds,
BenchmarkConditions(frameCount, warmupCount),
FrameInfo(cpuTotalTime / frameCount, gpuTotalTime / frameCount),
frames
frames,
)
} finally {
scene.close()

63
benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/benchmarks/lazygrid/LazyGrid.kt

@ -2,7 +2,6 @@ package benchmarks.lazygrid @@ -2,7 +2,6 @@ package benchmarks.lazygrid
import androidx.compose.foundation.gestures.scrollBy
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.lazy.grid.GridCells
@ -10,48 +9,32 @@ import androidx.compose.foundation.lazy.grid.LazyVerticalGrid @@ -10,48 +9,32 @@ import androidx.compose.foundation.lazy.grid.LazyVerticalGrid
import androidx.compose.foundation.lazy.grid.items
import androidx.compose.foundation.lazy.grid.rememberLazyGridState
import androidx.compose.material.Card
import androidx.compose.material.Checkbox
import androidx.compose.material.MaterialTheme
import androidx.compose.material.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.runtime.withFrameMillis
import androidx.compose.ui.Alignment
import androidx.compose.runtime.*
import androidx.compose.ui.Modifier
import androidx.compose.ui.semantics.contentDescription
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import kotlinx.coroutines.isActive
import kotlin.coroutines.suspendCoroutine
@Composable
fun LazyGrid() {
fun LazyGrid(smoothScroll: Boolean = false, withLaunchedEffectInItem: Boolean = false) {
val itemCount = 12000
val entries = remember {List(itemCount) { Entry("$it") }}
val state = rememberLazyGridState()
var smoothScroll by remember { mutableStateOf(false)}
MaterialTheme {
Column {
Row {
Checkbox(
checked = smoothScroll,
onCheckedChange = { value -> smoothScroll = value}
)
Text (text = "Smooth scroll", modifier = Modifier.align(Alignment.CenterVertically))
}
LazyVerticalGrid(
columns = GridCells.Fixed(4),
modifier = Modifier.fillMaxWidth().semantics { contentDescription = "IamLazy" },
state = state
) {
items(entries) {
ListCell(it)
ListCell(it, withLaunchedEffectInItem)
}
}
}
@ -62,35 +45,36 @@ fun LazyGrid() { @@ -62,35 +45,36 @@ fun LazyGrid() {
var direct by remember { mutableStateOf(true) }
if (smoothScroll) {
LaunchedEffect(Unit) {
while (smoothScroll) {
while (isActive) {
withFrameMillis { }
curItem = state.firstVisibleItemIndex
if (curItem == 0) direct = true
if (curItem > itemCount - 100) direct = false
state.scrollBy(if (direct) 5f else -5f)
state.scrollBy(if (direct) 55f else -55f)
}
}
} else {
LaunchedEffect(curItem) {
withFrameMillis { }
curItem += if (direct) 50 else -50
if (curItem >= itemCount) {
direct = false
curItem = itemCount - 1
} else if (curItem <= 0) {
direct = true
curItem = 0
LaunchedEffect(Unit) {
while(isActive) {
withFrameMillis {}
curItem += if (direct) 50 else -50
if (curItem >= itemCount) {
direct = false
curItem = itemCount - 1
} else if (curItem <= 0) {
direct = true
curItem = 0
}
state.scrollToItem(curItem)
}
state.scrollToItem(curItem)
}
}
}
data class Entry(val contents: String)
@Composable
private fun ListCell(entry: Entry) {
private fun ListCell(entry: Entry, withLaunchedEffect: Boolean = false) {
Card(
modifier = Modifier
.fillMaxWidth()
@ -102,5 +86,12 @@ private fun ListCell(entry: Entry) { @@ -102,5 +86,12 @@ private fun ListCell(entry: Entry) {
style = MaterialTheme.typography.h5,
modifier = Modifier.padding(16.dp)
)
if (withLaunchedEffect) {
LaunchedEffect(Unit) {
// Never resumed to imitate some async task running in an item's scope
suspendCoroutine { }
}
}
}
}

2
benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/benchmarks/example1/Clickable.kt → benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/benchmarks/multipleComponents/Clickable.kt

@ -17,7 +17,7 @@ @@ -17,7 +17,7 @@
// copy of https://github.com/JetBrains/compose-multiplatform-core/blob/d9e875b62e7bb4dd47b6b155d3a787251ff5bd38/compose/foundation/foundation/src/desktopMain/kotlin/androidx/compose/foundation/Clickable.desktop.kt#L78
// inefficient implementation of clickable for benchmarking purposes (Modifier.Node variant in AOSP is more optimized)
package benchmarks.example1
package benchmarks.multipleComponents
import androidx.compose.foundation.Indication
import androidx.compose.foundation.focusable

34
benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/benchmarks/example1/Example1.kt → benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/benchmarks/multipleComponents/MultipleComponents.kt

@ -16,7 +16,7 @@ @@ -16,7 +16,7 @@
// copy of https://github.com/JetBrains/compose-multiplatform-core/blob/d9e875b62e7bb4dd47b6b155d3a787251ff5bd38/compose/desktop/desktop/samples/src/jvmMain/kotlin/androidx/compose/desktop/examples/example1/Main.jvm.kt#L17
package benchmarks.example1
package benchmarks.multipleComponents
import androidx.compose.animation.animateColorAsState
import androidx.compose.animation.core.TweenSpec
@ -79,8 +79,6 @@ import androidx.compose.ui.Alignment @@ -79,8 +79,6 @@ import androidx.compose.ui.Alignment
import androidx.compose.ui.ExperimentalComposeUiApi
import androidx.compose.ui.Modifier
import androidx.compose.ui.focus.FocusRequester
import androidx.compose.ui.focus.FocusRequester.Companion.FocusRequesterFactory.component1
import androidx.compose.ui.focus.FocusRequester.Companion.FocusRequesterFactory.component2
import androidx.compose.ui.focus.focusProperties
import androidx.compose.ui.focus.focusRequester
import androidx.compose.ui.geometry.Offset
@ -146,7 +144,7 @@ import org.jetbrains.compose.resources.painterResource @@ -146,7 +144,7 @@ import org.jetbrains.compose.resources.painterResource
import kotlin.random.Random
@Composable
fun Example1() {
fun MultipleComponentsExample(isVectorGraphicsSupported: Boolean = true) {
val uriHandler = LocalUriHandler.current
MaterialTheme {
Scaffold(
@ -154,10 +152,12 @@ fun Example1() { @@ -154,10 +152,12 @@ fun Example1() {
TopAppBar(
title = {
Row(verticalAlignment = Alignment.CenterVertically) {
Image(
painterResource(Res.drawable.example1_sailing),
contentDescription = "Star"
)
if (isVectorGraphicsSupported) {
Image(
painterResource(Res.drawable.example1_sailing),
contentDescription = "Star"
)
}
Text("Desktop Compose Elements")
}
}
@ -184,7 +184,7 @@ fun Example1() { @@ -184,7 +184,7 @@ fun Example1() {
content = { innerPadding ->
Row(Modifier.padding(innerPadding)) {
LeftColumn(Modifier.weight(1f))
MiddleColumn(Modifier.width(500.dp))
MiddleColumn(Modifier.width(500.dp), isVectorGraphicsSupported = isVectorGraphicsSupported)
RightColumn(Modifier.width(200.dp))
}
}
@ -506,7 +506,7 @@ private fun ScrollableContent(scrollState: ScrollState) { @@ -506,7 +506,7 @@ private fun ScrollableContent(scrollState: ScrollState) {
}
@Composable
fun MiddleColumn(modifier: Modifier) = Column(modifier) {
fun MiddleColumn(modifier: Modifier, isVectorGraphicsSupported: Boolean) = Column(modifier) {
val (focusItem1, focusItem2) = FocusRequester.createRefs()
val text = remember {
mutableStateOf("Hello \uD83E\uDDD1\uD83C\uDFFF\u200D\uD83E\uDDB0")
@ -564,12 +564,14 @@ fun MiddleColumn(modifier: Modifier) = Column(modifier) { @@ -564,12 +564,14 @@ fun MiddleColumn(modifier: Modifier) = Column(modifier) {
Modifier.size(200.dp)
)
Icon(
painterResource(Res.drawable.example1_ic_call_answer),
"Localized description",
Modifier.size(100.dp).align(Alignment.CenterVertically),
tint = Color.Blue.copy(alpha = 0.5f)
)
if (isVectorGraphicsSupported) {
Icon(
painterResource(Res.drawable.example1_ic_call_answer),
"Localized description",
Modifier.size(100.dp).align(Alignment.CenterVertically),
tint = Color.Blue.copy(alpha = 0.5f)
)
}
}
Box(

2
benchmarks/multiplatform/benchmarks/src/desktopMain/kotlin/main.desktop.kt

@ -7,6 +7,6 @@ import kotlinx.coroutines.Dispatchers @@ -7,6 +7,6 @@ import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.runBlocking
fun main(args: Array<String>) {
Args.parseArgs(args)
Config.setGlobalFromArgs(args)
runBlocking(Dispatchers.Main) { runBenchmarks() }
}

2
benchmarks/multiplatform/benchmarks/src/iosMain/kotlin/main.ios.kt

@ -7,7 +7,7 @@ import kotlinx.coroutines.MainScope @@ -7,7 +7,7 @@ import kotlinx.coroutines.MainScope
import kotlinx.coroutines.launch
fun main(args : List<String>) {
Args.parseArgs(args.toTypedArray())
Config.setGlobalFromArgs(args.toTypedArray())
MainScope().launch {
runBenchmarks(graphicsContext = graphicsContext())
println("Completed!")

2
benchmarks/multiplatform/benchmarks/src/macosMain/kotlin/main.macos.kt

@ -5,6 +5,6 @@ @@ -5,6 +5,6 @@
import kotlinx.coroutines.runBlocking
fun main(args : Array<String>) {
Args.parseArgs(args)
Config.setGlobalFromArgs(args)
runBlocking { runBenchmarks(graphicsContext = graphicsContext()) }
}

74
benchmarks/multiplatform/benchmarks/src/wasmJsMain/kotlin/main.wasmJs.kt

@ -1,15 +1,81 @@ @@ -1,15 +1,81 @@
@file:OptIn(ExperimentalJsExport::class)
import kotlinx.browser.window
import kotlinx.coroutines.MainScope
import kotlinx.coroutines.launch
import kotlinx.coroutines.*
import org.w3c.dom.url.URLSearchParams
import kotlin.js.Promise
val jsOne = 1.toJsNumber()
fun main(args: Array<String>) {
if (isD8env().toBoolean()) {
mainD8(args)
} else {
mainBrowser()
}
}
fun main() {
fun mainBrowser() {
val urlParams = URLSearchParams(window.location.search.toJsString())
var i = 0
val args = generateSequence { urlParams.get("arg${i++}") }.toList().toTypedArray()
Args.parseArgs(args)
Config.setGlobalFromArgs(args)
MainScope().launch {
runBenchmarks()
println("Completed!")
}
}
// Currently, the initialization can't be adjusted to avoid calling the fun main, but
// we don't want use the default fun main, because Jetstream3 requires running the workloads separately / independently of each other.
// Also, they require that a benchmark completes before the function exists, which is not possible with if they just call fun main.
// Therefore, they'll rely on fun customLaunch, which returns a Promise (can be awaited for).
fun mainD8(args: Array<String>) {
println("mainD8 is intentionally doing nothing. Read the comments in main.wasmJs.kt")
}
private val basicConfigForD8 = Config(
// Using only SIMPLE mode, because VSYNC_EMULATION calls delay(...),
// which is implemented via setTimeout on web targets.
// setTimeout implementation is simplified in D8, making
// the VSYNC_EMULATION mode meaningless when running in D8
modes = setOf(Mode.SIMPLE),
// MultipleComponents is unsupported, because it uses vector icons. D8 doesn't provide XML parsing APIs.
// But there is an alternative workload called 'MultipleComponents-NoVectorGraphics'
disabledBenchmarks = setOf("MultipleComponents"),
)
@JsExport
fun customLaunch(benchmarkName: String, frameCount: Int): Promise<JsAny?> {
val config = basicConfigForD8.copy(
benchmarks = mapOf(benchmarkName to frameCount)
)
Config.setGlobal(config)
return MainScope().promise {
runBenchmarks(warmupCount = 0)
jsOne
}
}
@JsExport
fun d8BenchmarksRunner(args: String): Promise<JsAny?> {
val config = Args.parseArgs(args.split(" ").toTypedArray())
.copy(
modes = basicConfigForD8.modes,
disabledBenchmarks = basicConfigForD8.disabledBenchmarks
)
Config.setGlobal(config)
return MainScope().promise {
runBenchmarks()
jsOne
}
}
private fun isD8env(): JsBoolean =
js("typeof isD8 !== 'undefined'")

13
benchmarks/multiplatform/benchmarks/src/wasmJsMain/resources/launcher.mjs

@ -0,0 +1,13 @@ @@ -0,0 +1,13 @@
globalThis.isD8 = true;
import * as skiko from './skikod8.mjs';
import { instantiate } from './compose-benchmarks-benchmarks-wasm-js.uninstantiated.mjs';
const exports = (await instantiate({
'./skiko.mjs': skiko
})).exports;
await import('./polyfills.mjs');
await exports.d8BenchmarksRunner(Array.from(arguments).join(' '));
console.log('Finished');

24
benchmarks/multiplatform/benchmarks/src/wasmJsMain/resources/launcher_jetstream3.mjs

@ -0,0 +1,24 @@ @@ -0,0 +1,24 @@
globalThis.isD8 = true;
import * as skiko from './skikod8.mjs';
import { instantiate } from './compose-benchmarks-benchmarks-wasm-js.uninstantiated.mjs';
const exports = (await instantiate({
'./skiko.mjs': skiko
})).exports;
await import('./polyfills.mjs');
/*
AnimatedVisibility,
LazyGrid,
LazyGrid-ItemLaunchedEffect,
LazyGrid-SmoothScroll,
LazyGrid-SmoothScroll-ItemLaunchedEffect,
VisualEffects,
MultipleComponents-NoVectorGraphics
*/
let name = arguments[0] ? arguments[0] : 'AnimatedVisibility';
let frameCount = arguments[1] ? parseInt(arguments[1]) : 1000;
await exports.customLaunch(name, frameCount);
console.log('Finished');

56
benchmarks/multiplatform/benchmarks/src/wasmJsMain/resources/polyfills.mjs

@ -0,0 +1,56 @@ @@ -0,0 +1,56 @@
// polyfills.mjs
if (typeof globalThis.window === 'undefined') {
globalThis.window = globalThis;
}
if (typeof globalThis.navigator === 'undefined') {
globalThis.navigator = {};
}
if (!globalThis.navigator.languages) {
globalThis.navigator.languages = ['en-US', 'en'];
globalThis.navigator.userAgent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36';
globalThis.navigator.platform = "MacIntel";
}
if (!globalThis.gc) {
// No GC control in D8
globalThis.gc = () => {
// console.log('gc called');
};
}
// Minimal Blob polyfill
class BlobPolyfill {
constructor(uint8, type = '') {
this._uint8 = uint8;
this._type = type;
}
get size() {
return this._uint8.byteLength;
}
get type() { return this._type; }
async arrayBuffer() {
console.log('arrayBuffer called');
return this._uint8.buffer;
}
}
globalThis.fetch = async (p) => {
let data;
try {
let path = p.replace(/^\.\//, '');
console.log('fetch', path);
data = read(path, 'binary');
} catch (err) {
console.log('error', err);
}
const uint8 = new Uint8Array(data);
return {
ok: true,
status: 200,
async blob() {
return new BlobPolyfill(uint8, 'application/xml');
},
};
};
Loading…
Cancel
Save