Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-71185

race condition in creation of `master.key`

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • core

      DefaultConfidentialStore is not thread safe

      Observed in https://github.com/jenkinsci/reverse-proxy-auth-plugin/pull/94/checks?check_run_id=13195833273

      and reproduced by setting a breakpoint on DefaulConfidentialStore<init> suspending the current Thread only and running the unit test from the PR.

      If the master.key is not present all callers will try and create and persist it -

      Some may fail in the AtomicFile.commit and some may "succeed" overwriting an existing file.

      The former can block initial startup whereas the later would silently work however different parts of the codebase would have a different idea of what the master.key is and so on first restart some things may not be able to be recovered.

      There could also be issues for things such as config-as code.

      In the case of the discovering PR the 2 threads that are creating the master.key at the same time are:

      Daemon Thread [NullIdDescriptorMonitor.verify] (Suspended (breakpoint at line 66 in DefaultConfidentialStore))	
      	owns: HexStringConfidentialKey  (id=510)	
      	owns: ExtensionList$Lock  (id=511)	
      	DefaultConfidentialStore.<init>(File) line: 66	
      	DefaultConfidentialStore.<init>() line: 51	
      	ConfidentialStore.get() line: 92	
      	HexStringConfidentialKey.get() line: 47	
      	DefaultCrumbIssuer$DescriptorImpl.<init>() line: 127	
      	DefaultCrumbIssuer$DescriptorImpl$$FastClassByGuice$$138428620.GUICE$TRAMPOLINE(int, Object, Object[]) line: not available	
      	DefaultCrumbIssuer$DescriptorImpl$$FastClassByGuice$$138428620.apply(Object, Object) line: not available	
      	DefaultConstructionProxyFactory$FastClassProxy<T>.newInstance(Object...) line: 82	
      	ConstructorInjector<T>.provision(InternalContext, ConstructionContext<T>) line: 114	
      	ConstructorInjector<T>.access$000(ConstructorInjector, InternalContext, ConstructionContext) line: 33	
      	ConstructorInjector$1.call() line: 98	
      	ProvisionListenerStackCallback$Provision.provision() line: 109	
      	ExtensionFinder$GuiceFinder$SezpozModule.onProvision(ProvisionInvocation<T>) line: 568	
      	ProvisionListenerStackCallback$Provision.provision() line: 117	
      	ProvisionListenerStackCallback<T>.provision(InternalContext, ProvisionCallback<T>) line: 66	
      	ConstructorInjector<T>.construct(InternalContext, Dependency<?>, ProvisionListenerStackCallback<T>) line: 93	
      	ConstructorBindingImpl$Factory<T>.get(InternalContext, Dependency<?>, boolean) line: 296	
      	ProviderToInternalFactoryAdapter<T>.get() line: 40	
      	SingletonScope$1.get() line: 169	
      	ExtensionFinder$GuiceFinder$FaultTolerantScope$1.get() line: 444	
      	InternalFactoryToProviderAdapter<T>.get(InternalContext, Dependency<?>, boolean) line: 45	
      	InjectorImpl$1.get() line: 1100	
      	ExtensionFinder$GuiceFinder._find(Class<U>, List<ExtensionComponent<U>>, Injector) line: 402	
      	ExtensionFinder$GuiceFinder.find(Class<U>, Hudson) line: 393	
      	ClassicPluginStrategy.findComponents(Class<T>, Hudson) line: 358	
      	ExtensionList<T>.load() line: 384	
      	ExtensionList<T>.ensureLoaded() line: 320	
      	ExtensionList<T>.iterator() line: 172	
      	NullIdDescriptorMonitor.verify() line: 72	
      	NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]	
      	NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62	
      	DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43	
      	Method.invoke(Object, Object...) line: 498	
      	InitializerFinder(TaskMethodFinder<T>).invoke(Method) line: 109	
      	TaskMethodFinder$TaskImpl.run(Reactor) line: 185	
      	Jenkins$5(Reactor).runTask(Task) line: 305	
      	Jenkins$5.runTask(Task) line: 1158	
      	Reactor$2.run() line: 222	
      	Reactor$Node.run() line: 121	
      	ImpersonatingExecutorService$1.run() line: 68	
      	ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1149	
      	ThreadPoolExecutor$Worker.run() line: 624	
      	Thread.run() line: 750	
      

      and

      Daemon Thread [Loading global config] (Suspended (breakpoint at line 66 in DefaultConfidentialStore))	
      	owns: CryptoConfidentialKey  (id=482)	
      	DefaultConfidentialStore.<init>(File) line: 66	
      	DefaultConfidentialStore.<init>() line: 51	
      	ConfidentialStore.get() line: 92	
      	CryptoConfidentialKey.getKey() line: 35	
      	CryptoConfidentialKey.decrypt() line: 130	
      	HistoricalSecrets.decrypt(String, CryptoConfidentialKey) line: 55	
      	Secret.decrypt(String) line: 212	
      	Secret.fromString(String) line: 252	
      	ReverseProxySecurityRealm.readResolve() line: 399	
      	NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]	
      	NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62	
      	DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43	
      	Method.invoke(Object, Object...) line: 498	
      	SerializationMembers.callReadResolve(Object) line: 78	
      	RobustReflectionConverter.unmarshal(HierarchicalStreamReader, UnmarshallingContext) line: 290	
      	ReferenceByXPathUnmarshaller(TreeUnmarshaller).convert(Object, Class, Converter) line: 74	
      	ReferenceByXPathUnmarshaller(AbstractReferenceUnmarshaller).convert(Object, Class, Converter) line: 72	
      	ReferenceByXPathUnmarshaller(TreeUnmarshaller).convertAnother(Object, Class, Converter) line: 68	
      	RobustReflectionConverter.unmarshalField(UnmarshallingContext, Object, Class, Field) line: 454	
      	RobustReflectionConverter.doUnmarshal(Object, HierarchicalStreamReader, UnmarshallingContext) line: 350	
      	RobustReflectionConverter.unmarshal(HierarchicalStreamReader, UnmarshallingContext) line: 289	
      	ReferenceByXPathUnmarshaller(TreeUnmarshaller).convert(Object, Class, Converter) line: 74	
      	ReferenceByXPathUnmarshaller(AbstractReferenceUnmarshaller).convert(Object, Class, Converter) line: 72	
      	ReferenceByXPathUnmarshaller(TreeUnmarshaller).convertAnother(Object, Class, Converter) line: 68	
      	ReferenceByXPathUnmarshaller(TreeUnmarshaller).convertAnother(Object, Class) line: 52	
      	ReferenceByXPathUnmarshaller(TreeUnmarshaller).start(DataHolder) line: 136	
      	ReferenceByXPathMarshallingStrategy(AbstractTreeMarshallingStrategy).unmarshal(Object, HierarchicalStreamReader, DataHolder, ConverterLookup, Mapper) line: 32	
      	XStream2(XStream).unmarshal(HierarchicalStreamReader, Object, DataHolder) line: 1421	
      	XStream2.unmarshal(HierarchicalStreamReader, Object, DataHolder, boolean) line: 189	
      	XStream2.unmarshal(HierarchicalStreamReader, Object, DataHolder) line: 160	
      	XStream2(XStream).unmarshal(HierarchicalStreamReader, Object) line: 1399	
      	XmlFile.unmarshal(Object, boolean) line: 196	
      	XmlFile.unmarshal(Object) line: 179	
      	Hudson(Jenkins).loadConfig() line: 3296	
      	Jenkins.access$1200(Jenkins) line: 340	
      	Jenkins$12.run(Reactor) line: 3398	
      	TaskGraphBuilder$TaskImpl.run(Reactor) line: 175	
      	Jenkins$5(Reactor).runTask(Task) line: 305	
      	Jenkins$5.runTask(Task) line: 1158	
      	Reactor$2.run() line: 222	
      	Reactor$Node.run() line: 121	
      	ImpersonatingExecutorService$1.run() line: 68	
      	ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1149	
      	ThreadPoolExecutor$Worker.run() line: 624	
      	Thread.run() line: 750	
      

      that was shown in the debugger - but in either case the obtaining of the next random bytes could block depending on the secure random implementation thus the race condition period can be longer than it initially appears.

            Unassigned Unassigned
            teilo James Nord
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: