Does it work for 5 minutes?

(Aside: Keep in mind that such aggressive scanning will get you rate limited by Bitbucket Cloud, so you probably want to set up webhook notification)

If it works for 5 minutes but not for 2 then that could well be a bug in the cloudbees-folders plugin.

I've tried also with 5 and 10 minutes, neither works for me.
Scan Multibranch Pipeline Log is not being updated, shows only information from last scan.

Can you tell me more about rate limit? Isn't this just fetching refs?

If this scanning is so aggressive, is there any other setting to just fetch known branches? All I really want to achieve, is automated build of master branch after commit. It should be possible to do without webhooks, I guess.

So I just tried on my own server,

And as you can see the periodically runs, the initial scan was 9:32 and then when I check again it runs at 9:34... and then again at 9:36

now there is a bug, if I recall correctly, where the 1 minute interval gets checked against 60 seconds and hence will almost always be 60 seconds since last check and so only every other minute will it run... but if I understand correctly that should only apply to the 1 minute interval and not affect the larger intervals (it's due to the cron tasks running once per minute)

But the tasks are running for me.

I'd check your master to see if something is blocking either the Jenkins cron thread or one of the jenkins.util.Timer threads.

Additionally, there is a limit of 5 concurrent indexing/scanning tasks: https://github.com/jenkinsci/cloudbees-folder-plugin/blob/master/src/main/java/com/cloudbees/hudson/plugins/folder/computed/ThrottleComputationQueueTaskDispatcher.java#L49 so if you have other jobs that are say indexing GitHub and they have hit the GitHub rate limit, your indexing may just be blocked waiting for those to complete

I did manage to get the schedule somehow working on Jenkins 2.60.3 on docker image.

However it's running every ~12 minutes, despite being set to 5 minute interval.

I also have 2.79 on my mac and as docker image, and the schedule isn't working on either of them.

UPDATE

With 2 minute interval, indexing is done every 3 minutes.

Maybe related to https://github.com/jenkinsci/cloudbees-folder-plugin/pull/108? Can you see if that patch works?

After updating all of my plugins early last week, I'm now also seeing what appears to be at least a variant of this issue. 

We have a GitHub Organization folder setup, which correctly scans our entire org. Only one of our repositories at the moment contains Jenkins files, and it used to correctly scan all branches within that repository and pickup them up. After updating last week, the branches are no longer scanned. When I check the organization scan log, I see this:

Examining
our_org_name/our_repo_name
Checking branches... Getting remote branches... Checking branch
bug/branch_name_here
‘Jenkinsfile’ found Met criteria 1 branches were processed (query completed) 1 branches were processed Finished examining our_org_name/our_repo_name

...but we have probably 15 branches with a Jenkinsfile in them. It just so happens to be that the only branch it scanned is alphabetically the first in our repo. Going into the repository and looking at it's scan log, the scan does not happen at all. I don't recall in this setup if it is the organization or repository level scan that normally would have kicked off our build. I tested making a commit to the one branch that the org scan does detect, and that did not initiate a repository level scan. 

jglick if you have any suggestions of things to test/try/log or what not I'm all ears. This is having a big impact on our teams workflow.  I'll keep digging, but I suspect this is the same issue as described above as the repository level of the GitHub Org folder appears to be the same thing as a multibranch pipeline project. 

Also worth nothing, when viewing the config at the repository level it has.a scan interval set at 1 day. Since this config is created by the GitHub Org folder, we can't configure it. But despite that, it's not actually running once a day. I'm looking at it right now and the last scan was when we manually kicked it off 2 days ago. 

Actually, while monitoring the log I saw this when the scan ran:

WARNING: JENKINS-45892: reference to jenkins.branch.OrganizationFolder@34cac973[SC] being saved from unexpected /var/lib/jenkins/jobs/SC/computation/computation.xml
java.lang.IllegalStateException
 at hudson.XmlFile.replaceIfNotAtTopLevel(XmlFile.java:223)
 ...
 at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.save(FolderComputation.java:208)
 at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:192)
 at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:911)
 at hudson.model.ResourceController.execute(ResourceController.java:97)
 at hudson.model.Executor.run(Executor.java:429)

Not sure yet where that puts my issue in relation to the originally reported one, but still commenting in the hope it helps...

hardwickj

So I found this in one of my computation.xml files:

<?xml version='1.0' encoding='UTF-8'?>
<jenkins.branch.OrganizationFolder_-OrganizationScan plugin="branch-api@2.0.17">
<actions>
<hudson.model.CauseAction>
<causeBag class="linked-hash-map">
<entry>
<hudson.triggers.TimerTrigger_-TimerTriggerCause/>
<int>1</int>
</entry>
</causeBag>
</hudson.model.CauseAction>
<io.jenkins.blueocean.service.embedded.BlueOceanUrlAction plugin="blueocean-rest-impl@1.3.5">
<blueOceanUrlObject class="io.jenkins.blueocean.service.embedded.BlueOceanUrlObjectImpl">
<mappedUrl>blue/organizations/jenkins/pipelines/</mappedUrl>
<modelObject class="hudson.model.Queue$WaitingItem">
<actions>
<hudson.model.CauseAction reference="../../../../../hudson.model.CauseAction"/>
<io.jenkins.blueocean.service.embedded.BlueOceanUrlAction reference="../../../.."/>
</actions>
<id>15319</id>
<task class="jenkins.branch.OrganizationFolder" resolves-to="hudson.model.AbstractItem$Replacer" plugin="branch-api@2.0.17">
<fullName>cloudbeers</fullName>
</task>
<inQueueSince>1515907380276</inQueueSince>
<timestamp>
<time>1515907380276</time>
<timezone>Etc/UTC</timezone>
</timestamp>
</modelObject>
</blueOceanUrlObject>
</io.jenkins.blueocean.service.embedded.BlueOceanUrlAction>
<io.jenkins.blueocean.service.embedded.BlueOceanUrlAction reference="../io.jenkins.blueocean.service.embedded.BlueOceanUrlAction"/>
</actions>
<result>
<name>SUCCESS</name>
<ordinal>0</ordinal>
<color>BLUE</color>
<completeBuild>true</completeBuild>
</result>
<durations>
<long>21123</long>
</durations>
<timestamp>1515907380282</timestamp>
<duration>21123</duration>
</jenkins.branch.OrganizationFolder_-OrganizationScan>

That smells very fishy... I go digging further and find: https://issues.jenkins-ci.org/browse/JENKINS-48322 so hardwickj that looks to be the issue you had and upgrading to BlueOcean 1.4 should fix the strange log messages for you

hardwickj

When I check the organization scan log, I see this:

Examining
our_org_name/our_repo_name
Checking branches... Getting remote branches... Checking branch
bug/branch_name_here
‘Jenkinsfile’ found Met criteria 1 branches were processed (query completed) 1 branches were processed Finished examining our_org_name/our_repo_name

...but we have probably 15 branches with a Jenkinsfile in them. It just so happens to be that the only branch it scanned is alphabetically the first in our repo.

So the Organization Folder just needs to know that there is at least one branch in the repository in order to create the child multibranch project. Once the child multibranch project has been created the responsibility for building belongs to the child.

The organization folder is designed not to burn excessive rate limit checking unnecessary branches (it could be even smarter, e.g. by peeking inside an existing multibranch project and checking the branches in order of most recent build rather than alpha order, but that's a separate issue) 

Now recently we fixed a bug in Organization Folder, namely there was a side-effect where every scan of an organization folder would always trigger a scan of all the child Multibranch projects (the fix was in JENKINS-48536). The fixed behaviour only triggers the child Multibranch project if the multibranch project was disabled (because there are no more branches with marker files) and has been re-enabled (e.g. at least one branch with a marker file has been introduced). The multibranch project being disabled will ignore all push events and not run periodic scans, so the organization folder needs to take over that responsibility. The bug was that previously it was being over-zealous.

Now the question is whether the per-repository scans are taking place or not... but that is really a question for a new issue.

In short hardwickj could you create a new JIRA issue for the issue you are seeing with the multibranch projects in an org folder not being periodically scanned.

FYI I just checked my own personal Jenkins instance, here's an org folder and it's most recent scan:

And here's a child multibranch project and it's most recent scan:

Notice that the times are completely unrelated (which is correct and proper because Periodically unless otherwise ran explicitly does not let you pick the time and instead randomizes the time to level the load)

Thus hardwickj the only real question is was that 1 day or 2 days since either of those ran... if it is 1 day then I think we can close this issue as fixed (after confirming that 1 minute does scan every minute... need to use a git repo for that though, otherwise the rate limit will get in the way)

The original one I was referring to last had a commit on a Friday later afternoon. I was reporting it here on Sunday early afternoon, so almost 2 days. 

New ticket for my issue created at JENKINS-48960

We're seeing the same issue on our Jenkins installation, version 2.89.4. After updating Jenkins and all plugins to latest versions this week (it was actually an old installation), the periodic organization scan stopped triggering at all.

I can still manually kickstart this scan, and it works as expected then, but for some reason it doesn't run on its own - and I have no idea how to debug it. If instructed, I will be happy to provide any debugging information.

Also seeing this issue on Jenkins ver. 2.89.4 from a clean docker install. All plugins are up to date. The timer never seems to trigger the scan but manual scans work fine.

So I think what has happened is that we have uncovered a bug in fixing another bug.

Up until "recently" the branch-api plugin's OrganizationFolder functionality would end up triggering a rescan of all child multibranch projects whenever it actually did a rescan. This was a bug, because the intent would be, if you setup the org scan as daily and the multibranch as daily (ignore that you cannot configure the multibranch's period at present) then you don't want all the scans at the same time in the day, rather you want them leveled throughout the day so that if you are using an SCM with an API rate limit, the rate limit usage is leveled across the whole day. By the organization folder triggering the scans of all child multibranch projects, effectively the rate limit usage would be spiked at one time in the day, which is actually a bad thing.

So what has made this bug surface is this change which fixed the bug whereby the Organization folder would always trigger a rescan of the child multibranch rather than only triggering a rescan if the scan configuration has changed.

Because the Organization folder was no longer driving periodic scans, we now see the bug - which I suspect was always present - of the Multibranch's periodic setting not actually being applied.

On my own Jenkins the periodic setting is applied on at least some projects, so this is somewhat inconsistent, but it is incredibly annoying.

What I do find interesting is this one:

There is more than 24h since that one ran. It should run once every 24h, but it seems to be running every 48h.

Hmmm H H * * * is the crontab... so if that picks any time before 19:49 then yesterday it would have been less than 24h since the last run and so it would only pick up the run today... presumably it will run at 19:49 today but the last triggered field will be out my some number of milliseconds and so effectively it runs every second interval

https://github.com/jenkinsci/cloudbees-folder-plugin/pull/119 might fix the incidental issue of "every minute" not running every minute... but it would be really good to understand how others are not getting the child multibranch scan being triggered at least once every two days?

OR is it being triggered once every two days, but the org folder is set to a higher interval and thus "longer than every 2 days" is being translated into "never" because people are manually triggering as a workaround to being unable to configure the child triggers

We have to scan all our multibranch pipeline repositories separately and cannot configure them (yet) as stated above...
Otherwise the branch and PR jobs never get triggered.

Also the new Bitbucket webhooks are not supported (JENKINS-47891) yet so we are kind of stuck  as we cannot install extra plugins on Bitbucket Server.
This is really annoying.

As a workaround, we created a pipeline job as follows:

import org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject;
node {
    Jenkins.instance.items
    .findAll { it instanceof WorkflowMultiBranchProject }
    .each {
        echo it.name
        it.indexing.run()
    }
}

I set it to trigger to build periodically, every two minutes: H/2 * * * *

But, I had to do script approvals.  This is temporary for us until this bug gets fixed, after which I'll remove this job and the script approvals.

stephenconnolly, I noticed this was marked as resolved. There are no comments regarding the fix or release. Do you know when we can expect to pick up the new plugin version? And what that version will be?

Thanks in advance!

Derek

dsudduth I think it is released as 6.7 - https://plugins.jenkins.io/cloudbees-folder

Thanks Aurimas, looks like it's the subdirectories (child jobs) that still exhibit the issue. That's what I'm experiencing after reading Stephen's comments further.

Subfolders have their own period of scanning that is not the period of the parent

stephenconnolly - You're correct. There are settings generated for each repo where a Jenkinsfile is detected. However, there is an issue with permissions when dynamically generating jobs. The settings are viewable, but permissions are denied. Separate issue, I know. But, it does make it challenging to modify the scan interval.

Looks like more of a global issue outside the scope of this plugin. It is also seen in the Github plugin.

https://stackoverflow.com/questions/48654556/jenkins-how-to-set-the-periodic-scan-interval-of-jobs-under-github-organization