8000 Exception on reading MemoryUsage (possible wrong max memory value) · Issue #6814 · oracle/graal · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exception on reading MemoryUsage (possible wrong max memory value) #6814

Closed
joggeli34 opened this issue Jun 16, 2023 · 7 comments
Closed

Exception on reading MemoryUsage (possible wrong max memory value) #6814

joggeli34 opened this issue Jun 16, 2023 · 7 comments
Assignees
@oubidar-Abderrahim
Labels
bug

Comments

@joggeli34
Copy link

Describe the issue
With the newest graalvm release, it is now possible to read out the memory usage. We use quarkus with micrometer, where we get following exception:

2023-06-15 14:18:58,409 WARN  [io.mic.cor.ins.int.DefaultGauge] (executor-thread-3) Failed to apply the value function for the gauge 'jvm.memory.used'. Note that subsequent logs will be logged at debug level.: java.lang.IllegalArgumentException: committed = 8573528 should be < max = 8388608
	at java.management@17.0.7/java.lang.management.MemoryUsage.<init>(MemoryUsage.java:166)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.genscavenge.AbstractMemoryPoolMXBean.memoryUsage(AbstractMemoryPoolMXBean.java:75)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.genscavenge.GenScavengeMemoryPoolMXBeans$EdenMemoryPoolMXBean.getUsage(GenScavengeMemoryPoolMXBeans.java:94)
	at io.micrometer.core.instrument.binder.jvm.JvmMemory.getUsage(JvmMemory.java:76)
	at io.micrometer.core.instrument.binder.jvm.JvmMemory.getUsageValue(JvmMemory.java:66)
	at io.micrometer.core.instrument.binder.jvm.JvmMemoryMetrics.lambda$bindTo$0(JvmMemoryMetrics.java:82)
	at io.micrometer.core.instrument.internal.DefaultGauge.value(DefaultGauge.java:53)
	at io.micrometer.prometheus.PrometheusMeterRegistry.lambda$newGauge$5(PrometheusMeterRegistry.java:331)
	at io.micrometer.prometheus.MicrometerCollector.collect(MicrometerCollector.java:75)
	at io.prometheus.client.Collector.collect(Collector.java:45)
	at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:204)
	at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:219)
	at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:152)
	at io.prometheus.client.exporter.common.TextFormat.write004(TextFormat.java:71)
	at io.prometheus.client.exporter.common.TextFormat.writeFormat(TextFormat.java:53)
	at io.micrometer.prometheus.PrometheusMeterRegistry.scrape(PrometheusMeterRegistry.java:154)
	at io.micrometer.prometheus.PrometheusMeterRegistry.scrape(PrometheusMeterRegistry.java:149)
	at io.micrometer.prometheus.PrometheusMeterRegistry.scrape(PrometheusMeterRegistry.java:120)
	at io.quarkus.micrometer.runtime.export.handlers.PrometheusHandler.doHandle(PrometheusHandler.java:63)
	at io.quarkus.micrometer.runtime.export.handlers.PrometheusHandler.handle(PrometheusHandler.java:43)
	at io.quarkus.micrometer.runtime.export.handlers.PrometheusHandler.handle(PrometheusHandler.java:18)
	at io.vertx.ext.web.impl.BlockingHandlerDecorator.lambda$handle$0(BlockingHandlerDecorator.java:48)
	at io.vertx.core.impl.ContextBase.lambda$null$0(ContextBase.java:137)
	at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264)
	at io.vertx.core.impl.ContextBase.lambda$executeBlocking$1(ContextBase.java:135)
	at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:576)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base@17.0.7/java.lang.Thread.run(Thread.java:833)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:807)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:210)

We used as run parameters "-Xmx30m -Xmn10m". So there is also a difference between the 10m and the 8m reported here.

Steps to reproduce the issue
We don't have a reproducer yet separate from our production code.

Describe GraalVM and your environment:

  • GraalVM version CE 17.0
  • JDK major version: 17
  • OS: Linux
  • Architecture: AMD64
  • Quarkus: 3.1.1

More details
Additionally, we saw that the eden-space max value is not always readed out as the same, but we are not sure if this also could be some aggregation issue (with setting -Xmx100m):
image
@joggeli34 joggeli34 added the bug label Jun 16, 2023
@korkutkose
Copy link

Same here with Spring Boot 3.1.0 built with graalvm-jdk-17.
We did not specify any xms or xmx values while running the app on k8s with resource request: 128m and limit 256m.

2023-06-21 07:58:20,657 INFO [main,,] o.s.b.StartupInfoLogger: Started Application in 3.801 seconds (process running for 3.809)
24
2023-06-21 07:58:24,681 INFO [http-nio-9099-exec-1,,] o.a.j.l.DirectJDKLog: Initializing Spring DispatcherServlet 'dispatcherServlet'
25
2023-06-21 08:05:07,399 WARN [http-nio-9099-exec-2,,] i.m.c.u.i.l.AbstractInternalLogger: Failed to apply the value function for the gauge 'jvm.memory.committed'. Note that subsequent logs will be logged at debug level.
26
java.lang.IllegalArgumentException: committed = 14680064 should be < max = 12582912

@oubidar-Abderrahim oubidar-Abderrahim self-assigned this Jun 26, 2023
@oubidar-Abderrahim
Copy link
Member

Hi, Thank you for reporting this issue, could you please provide a reproducer example to use in debugging the issue? Thank you

@petermz petermz mentioned this issue Jul 4, 2023
Merged
@oubidar-Abderrahim
Copy link
Member

Any updates on the reproducer?

@joggeli34
Copy link
Author

I haven't had the time to create a reproducer. I will try to find time next week.

@0xabadea
Copy link

Attaching a Spring Boot project that consistently reproduces the issue for me: issue6814.zip To reproduce:

  1. Unzip the attached zip file.
  2. Execute ./gradlew bootBuildImage
  3. In a separate terminal execute while true; do curl localhost:9080/actuator/prometheus; done.
  4. Execute docker run -m 256m -p 9080:9080 docker.io/library/issue6814:0.0.1-SNAPSHOT -Xmx128m -XX:MaxDirectMemorySize=40m

For me, the application starts up and immediately throws an exception very similar to the one in the original description:

2023-07-26T12:16:55.796Z  WARN [issue6814,10fcbf35e2e21545adde41bf5c61362a,44433b2bc18acabc] 1 --- [oundedElastic-1] i.m.c.instrument.internal.DefaultGauge   : Failed to apply the value function for the gauge 'jvm.memory.committed'. Note that subsequent logs will be logged at debug level.

java.lang.IllegalArgumentException: committed = 27262976 should be < max = 14680064
	at java.management@17.0.7/java.lang.management.MemoryUsage.<init>(MemoryUsage.java:166) ~[na:na]
	at com.oracle.svm.core.genscavenge.AbstractMemoryPoolMXBean.memoryUsage(AbstractMemoryPoolMXBean.java:75) ~[com.example.issue6814.Issue6814ApplicationKt:na]
	at com.oracle.svm.core.genscavenge.GenScavengeMemoryPoolMXBeans$EdenMemoryPoolMXBean.getUsage(GenScavengeMemoryPoolMXBeans.java:94) ~[na:na]
	at io.micrometer.core.instrument.binder.jvm.JvmMemory.getUsage(JvmMemory.java:76) ~[na:na]

Please note the exception is not identical: the original description refers to the jvm.memory.used gauge, while the sample project refers to jvm.memory.committed. However, both exceptions happen when calling GenScavengeMemoryPoolMXBeans$EdenMemoryPoolMXBean.getUsage, so hopefully the sample project is relevant.

@joggeli34
Copy link
Author

Any news about this? It still happens on graalvm jdk-21

@oubidar-Abderrahim
Copy link
Member

Should be fixed by #6930

This was referenced Nov 27, 2024
Closed
Closed
@zakkak zakkak mentioned this issue Dec 12, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@oubidar-Abderrahim oubidar-Abderrahim

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@0xabadea @korkutkose @oubidar-Abderrahim @joggeli34
0