Apache Ignite cache long running transaction

Apache Ignite cache long running transaction



Sometimes, not always, when the topology of my cluster changes it happens that my application hangs for up to 1 minute or more. In the log I then see the below Ignite warning. I guess this is the reason why my application hangs at a cache operation.



What is causing the long transaction? I expect it is either network issues or GC?



I wasn't able to find out which cache operation in my code is causing this long transaction. Does the warning help me find out what operation it is?


22:00:30.456 [grid-timeout-worker-#63][101] WARN org.apache.ignite.internal.diagnostic-[warning] Found long running transaction [startTime=21:58:57.176, curTime=22:00:30.456, tx=GridNearTxLocal [mappings=IgniteTxMappingsImpl , nearLocallyMapped=false, colocatedLocallyMapped=false, needCheckBackup=null, hasRemoteLocks=false, trackTimeout=false, lb=null, thread=<failed to find active thread 1498>, mappings=IgniteTxMappingsImpl , super=GridDhtTxLocalAdapter [nearOnOriginatingNode=false, nearNodes=, dhtNodes=, explicitLock=false, super=IgniteTxLocalAdapter [completedBase=null, sndTransformedVals=false, depEnabled=false, txState=IgniteTxStateImpl [activeCacheIds=, recovery=null, txMap=], super=IgniteTxAdapter [xidVer=GridCacheVersion [topVer=147994093, order=1536523104974, nodeOrder=74], writeVer=null, implicit=false, loc=true, threadId=1498, startTime=1536523137176, nodeId=e8153238-1d5a-4149-8db8-83a9fc820750, startVer=GridCacheVersion [topVer=147994093, order=1536523104974, nodeOrder=74], endVer=null, isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, timeout=0, sysInvalidate=false, sys=false, plc=2, commitVer=null, finalizing=NONE, invalidParts=null, state=ACTIVE, timedOut=false, topVer=AffinityTopologyVersion [topVer=-1, minorTopVer=0], duration=93280ms, onePhaseCommit=false], size=0]]]]



My caches are created like this:


<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="MFDB_JobList" />
<property name="cacheMode" value="PARTITIONED" />
<property name="backups" value="0" />
<property name="atomicityMode" value="TRANSACTIONAL"/>
<property name="writeSynchronizationMode" value="FULL_SYNC"/>
<property name="indexedTypes">
<list>
<value>java.util.UUID</value>
<value>CacheJobQueueEntry</value>
</list>
</property>
</bean>



The relevant ignite configuration looks like this:


<property name="networkTimeout" value="60000" />
<property name="networkSendRetryCount" value="10" />
<property name="failureDetectionTimeout" value="100000" />
<property name="clientFailureDetectionTimeout" value="100000" />






It sounds like exchange goes way too slow in your cluster. Could you please share logs and few consecutive thread dumps taken while application hangs?

– antkr
Sep 10 '18 at 10:10






@DonTequila How many nodes in the cluster do you have? How much data are you storing? Is persistence to disk enabled?

– Dmitriy
Sep 11 '18 at 18:45







@Dmitiy sorry for missing all this information. These errors already came with only 3 nodes. Persitance is emabled. I think I found out that I was querying a cache with lots of data using SqlQuery, this took quite long time on every cache update. Now I'm using SqlFieldsQuery with only a few columns and the whole cluster is must faster. Data thoughput was about average 500mb/s before, now only 40kb/s. I will keep an eye on it if the error still happens. Thanks!

– DonTequila
Sep 11 '18 at 21:14




1 Answer
1



go through below link:-



https://issues.apache.org/jira/browse/IGNITE-6980



And also try


<property name="atomicityMode" value="ATOMIC"/>



with Async Cache operation. It might help to minimize above issue.



Thanks for contributing an answer to Stack Overflow!



But avoid



To learn more, see our tips on writing great answers.



Required, but never shown



Required, but never shown




By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Popular posts from this blog

𛂒𛀶,𛀽𛀑𛂀𛃧𛂓𛀙𛃆𛃑𛃷𛂟𛁡𛀢𛀟𛁤𛂽𛁕𛁪𛂟𛂯,𛁞𛂧𛀴𛁄𛁠𛁼𛂿𛀤 𛂘,𛁺𛂾𛃭𛃭𛃵𛀺,𛂣𛃍𛂖𛃶 𛀸𛃀𛂖𛁶𛁏𛁚 𛂢𛂞 𛁰𛂆𛀔,𛁸𛀽𛁓𛃋𛂇𛃧𛀧𛃣𛂐𛃇,𛂂𛃻𛃲𛁬𛃞𛀧𛃃𛀅 𛂭𛁠𛁡𛃇𛀷𛃓𛁥,𛁙𛁘𛁞𛃸𛁸𛃣𛁜,𛂛,𛃿,𛁯𛂘𛂌𛃛𛁱𛃌𛂈𛂇 𛁊𛃲,𛀕𛃴𛀜 𛀶𛂆𛀶𛃟𛂉𛀣,𛂐𛁞𛁾 𛁷𛂑𛁳𛂯𛀬𛃅,𛃶𛁼

Crossroads (UK TV series)

ữḛḳṊẴ ẋ,Ẩṙ,ỹḛẪẠứụỿṞṦ,Ṉẍừ,ứ Ị,Ḵ,ṏ ṇỪḎḰṰọửḊ ṾḨḮữẑỶṑỗḮṣṉẃ Ữẩụ,ṓ,ḹẕḪḫỞṿḭ ỒṱṨẁṋṜ ḅẈ ṉ ứṀḱṑỒḵ,ḏ,ḊḖỹẊ Ẻḷổ,ṥ ẔḲẪụḣể Ṱ ḭỏựẶ Ồ Ṩ,ẂḿṡḾồ ỗṗṡịṞẤḵṽẃ ṸḒẄẘ,ủẞẵṦṟầṓế