[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content


Eclipse Community Forums

      Home
Home » Eclipse Projects » OM2M » OM2M - Missing data in NSCL
OM2M - Missing data in NSCL [message #1749730] Sat, 10 December 2016 07:33 Go to next message
Eclipse UserFriend
Hi,

i set up OM2M NSCL on my computer A (8 cores, 16GB RAM) and create simple application in NSCL applications.

I was testing OM2M solution with JMeter:
1. Sending 15k data from PC A, with automatic sending speed (if response time was higher, data was sent slower). All data was in H2.
2. Sending 15k from A, B, C and D with automatic sending speed (if response time was higher, data was sent slower). It took 30min for about 18k but some data in H2 was missing.
3. Sending 12k, 20k, 24k in 32k from A, B, C and D with unlimited sending speed (took about 2min). In H2 was 1/3 of sent data.

Dont know what is worong... Has OM2M some problems with accepting data from multiple clients at high speed? Its possible to parallelize some OM2M components? I also noticed that WEBGUI is realy unresponsive when OM2M contains more then 10k of data. Its realy hard to browse throught all contentInstances.

Thanks,
Matej

[Updated on: Sat, 10 December 2016 07:36] by Moderator

Re: OM2M - Missing data in NSCL [message #1749786 is a reply to message #1749730] Mon, 12 December 2016 04:28 Go to previous messageGo to next message
Eclipse UserFriend
Dear Matej,

Thank you for your feedback.
I opened the following bugs to track and solve these two issues:

Bug 509062 - Missing data from database when using multiple fast clients
https://bugs.eclipse.org/bugs/show_bug.cgi?id=509062

Bug 509061 - Web interface unresponsive with 10k of data
https://bugs.eclipse.org/bugs/show_bug.cgi?id=509061

Regards,
Mahdi
Re: OM2M - Missing data in NSCL [message #1750130 is a reply to message #1749786] Fri, 16 December 2016 05:00 Go to previous messageGo to next message
Eclipse UserFriend
Dear Matej,

You are using the term NSCL here.
Are you using OM2M v0.8 which is the implementation of ETSI M2M (deprecated by ETSI and replace by oneM2M) ?
Re: OM2M - Missing data in NSCL [message #1750131 is a reply to message #1750130] Fri, 16 December 2016 05:01 Go to previous messageGo to next message
Eclipse UserFriend
If yes then could you please run your tests on OM2M 1.0 (implementation of oneM2M standard) ?
Re: OM2M - Missing data in NSCL [message #1750208 is a reply to message #1750131] Sun, 18 December 2016 06:27 Go to previous messageGo to next message
Eclipse UserFriend
Yes, i am using OM2M v0.8 because when i started writing Master thesis was OM2M v0.8 out. I implemented all of my plugins with OM2M v0.8 support.

[Updated on: Sun, 18 December 2016 06:28] by Moderator

Re: OM2M - Missing data in NSCL [message #1750209 is a reply to message #1750208] Sun, 18 December 2016 06:39 Go to previous messageGo to next message
Eclipse UserFriend
To help me investigating this issue, could you please make a quick test with OM2M 1.0 ? I suggest you to use the last version:
OM2M-DEV-13112016

You can use this code example to quickly create AE, Container and contentInstance resources using JSON and HTTP:
https://wiki.eclipse.org/OM2M/one/App

By the way, there is no big changes between oneM2M and ETSI M2M APIs.
Re: OM2M - Missing data in NSCL [message #1750210 is a reply to message #1750208] Sun, 18 December 2016 06:41 Go to previous messageGo to next message
Eclipse UserFriend
"OM2M-DEV-13112016" Is available for download here:
http://wiki.eclipse.org/OM2M/Download
Re: OM2M - Missing data in NSCL [message #1750309 is a reply to message #1749730] Mon, 19 December 2016 14:07 Go to previous messageGo to next message
Eclipse UserFriend
I used JMeter and "OM2M-DEV-13112016". HTTP Request with 1000ms response time.

4PC:
-A: 4core, 1500 data per core. Sum:6000
-B: 2core, 1500 data per core. Sum:3000
-C: 2core, 1500 data per core. Sum:3000
-D: 2core, 1500 data per core. Sum:3000

After 17 mins, i opened H2 database and executed:
SELECT COUNT(*) FROM CIN; Result was 9480.

I tried to DATA in web app and got that exception:

[INFO] - org.eclipse.om2m.core.router.Router
Request handling in the current CSE: /in-cse/in-name/MY_SENSOR/DATA
[INFO] - org.eclipse.om2m.core.router.Router
ResourceController to be used [ContentInstanceController]
[EL Severe]: cache: 2016-12-19 19:43:09.672--ClientSession(1533855376)--MAX TIME 600 seconds EXCEEDED FOR WRITELOCKMANAGER WAIT.  Waiting on Entity type: org.eclipse.om2m.commons.entities.ContainerEntitywith pk: /in-cse/cnt-761411626 currently locked by thread: qtp71587369-76 with the following trace:
                atjava.lang.Object.wait(Native Method)
                atorg.eclipse.persistence.sessions.server.ConnectionPool.acquireConnection(ConnectionPool.java:159)
                atorg.eclipse.persistence.sessions.server.ServerSession.getAccessors(ServerSession.java:538)
                atorg.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:557)
                atorg.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:242)
                atorg.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:228)
                atorg.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectOneRow(DatasourceCallQueryMechanism.java:714)
                atorg.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRowFromTable(ExpressionQueryMechanism.java:2801)
                atorg.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectOneRow(ExpressionQueryMechanism.java:2754)
                atorg.eclipse.persistence.queries.ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:545)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1168)
                atorg.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:899)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1127)
                atorg.eclipse.persistence.queries.ReadObjectQuery.execute(ReadObjectQuery.java:431)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.internalExecuteQuery(AbstractSession.java:3214)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1804)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1786)
                atorg.eclipse.persistence.internal.indirection.NoIndirectionPolicy.valueFromQuery(NoIndirectionPolicy.java:326)
                atorg.eclipse.persistence.mappings.ForeignReferenceMapping.valueFromRowInternal(ForeignReferenceMapping.java:2294)
                atorg.eclipse.persistence.mappings.OneToOneMapping.valueFromRowInternal(OneToOneMapping.java:1860)
                atorg.eclipse.persistence.mappings.ForeignReferenceMapping.valueFromRow(ForeignReferenceMapping.java:2144)
                atorg.eclipse.persistence.mappings.ForeignReferenceMapping.readFromRowIntoObject(ForeignReferenceMapping.java:1471)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildAttributesIntoObject(ObjectBuilder.java:461)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.refreshObjectIfRequired(ObjectBuilder.java:4304)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:1039)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildWorkingCopyCloneNormally(ObjectBuilder.java:898)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObjectInUnitOfWork(ObjectBuilder.java:851)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:734)
                atorg.eclipse.persistence.internal.descriptors.ObjectBuilder.buildObject(ObjectBuilder.java:688)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.buildObject(ObjectLevelReadQuery.java:795)
                atorg.eclipse.persistence.queries.ReadObjectQuery.registerResultInUnitOfWork(ReadObjectQuery.java:885)
                atorg.eclipse.persistence.queries.ReadObjectQuery.executeObjectLevelReadQuery(ReadObjectQuery.java:552)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1168)
                atorg.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:899)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1127)
                atorg.eclipse.persistence.queries.ReadObjectQuery.execute(ReadObjectQuery.java:431)
                atorg.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1215)
                atorg.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2896)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1804)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1786)
                atorg.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1737)
                atorg.eclipse.persistence.internal.jpa.EntityManagerImpl.executeQuery(EntityManagerImpl.java:900)
                atorg.eclipse.persistence.internal.jpa.EntityManagerImpl.findInternal(EntityManagerImpl.java:843)
                atorg.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:719)
                atorg.eclipse.persistence.internal.jpa.EntityManagerImpl.find(EntityManagerImpl.java:588)
                atorg.eclipse.om2m.persistence.eclipselink.internal.dao.ContainerDAO.find(ContainerDAO.java:40)
                atorg.eclipse.om2m.persistence.eclipselink.internal.dao.ContainerDAO.find(ContainerDAO.java:1)
                atorg.eclipse.om2m.core.controller.ContentInstanceController.doCreate(ContentInstanceController.java:98)
                atorg.eclipse.om2m.core.controller.Controller.doRequest(Controller.java:76)
                atorg.eclipse.om2m.core.router.Router.doRequest(Router.java:224)
                atorg.eclipse.om2m.binding.http.RestHttpServlet.service(RestHttpServlet.java:130)
                atjavax.servlet.http.HttpServlet.service(HttpServlet.java:790)
                atorg.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
                atorg.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
                atorg.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:76)
                atjavax.servlet.http.HttpServlet.service(HttpServlet.java:790)
                atorg.eclipse.equinox.http.jetty.internal.HttpServerManager$InternalHttpServiceServlet.service(HttpServerManager.java:360)
                atorg.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
                atorg.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
                atorg.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
                atorg.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
                atorg.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
                atorg.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
                atorg.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
                atorg.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
                atorg.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
                atorg.eclipse.jetty.server.Server.handle(Server.java:370)
                atorg.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
                atorg.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
                atorg.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
                atorg.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
                atorg.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
                atorg.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
                atorg.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
                atorg.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
                atorg.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
                atorg.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
                atjava.lang.Thread.run(Unknown Source)
 (There is no English translation for this message.)
[EL Severe]: cache: 2016-12-19 19:43:09.681--ServerSession(2053940301)--Current object locks:
Thread : qtp71587369-76
Locked Object : org.eclipse.om2m.commons.entities.ContainerEntity@88037c5
PK: /in-cse/cnt-761411626
Depth : 1
Locked Object : org.eclipse.om2m.commons.entities.AeEntity@46d0a766
PK: /in-cse/CAE704876354
Depth : 1

End of locked objects.


I did another test:
Sent 10k data (4 cores, 2500 per core). Throughput at start was around 140s/s at end around 10/s. After i started sending again throughput was 7/s at beginning.

Hmm do you know what could be wrong? Am i sending to much/fast data?

[Updated on: Mon, 19 December 2016 14:08] by Moderator

Re: OM2M - Missing data in NSCL [message #1750366 is a reply to message #1750309] Tue, 20 December 2016 07:31 Go to previous messageGo to next message
Eclipse UserFriend
Hi Matej,

Our team is investigating this issue.
Thank your for the report.
Re: OM2M - Missing data in NSCL [message #1751200 is a reply to message #1750366] Thu, 05 January 2017 10:08 Go to previous messageGo to next message
Eclipse UserFriend
Dear Matej,

1) After investigation, we think that the you are checking the database while the data is being persisted in the database.
This could explain why you found only 9480/15000.
This could explain also the error encountered while retrieving the conntent instances from the DATA container.

2) We made recently several fixes in the master branch so we recommend to use this version. We are going to upload new binaries in the wiki soon.

3) You could also try to use H2 database in server mode if it feeds your requirements.
We have the following tutorial for this purpose:
https://wiki.eclipse.org/OM2M/one/Advanced_DB

Thank you for your valuable feedback.
Re: OM2M - Missing data in NSCL [message #1751208 is a reply to message #1751200] Thu, 05 January 2017 11:41 Go to previous message
Eclipse UserFriend
Hi,

no problem, thanks for investigating issue and your reply. If i understand correct, you think that data is missing because H2 write problem?
Previous Topic:Develop Applications
Next Topic:What is the purpose of configuring RequestIdentifier?
Goto Forum:
  


Current Time: Thu Mar 06 05:29:38 EST 2025

Powered by FUDForum. Page generated in 0.07041 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top

Sign up to our Newsletter

A fresh new issue delivered monthly