#35783 fixes an actions rerun bug. Due to this bug, some runs may be
incorrectly marked as `StatusWaiting` even though all the jobs are in
done status. These runs cannot be run or cancelled. This PR adds a new
doctor command to fix the inconsistent run status.
```
gitea doctor check --run fix-actions-unfinished-run-status --fix
```
Thanks to @ChristopherHX for the test.
Fix#35781, #27472
The PR will not correct the wrong numbers automatically.
There is a cron task `check_repo_stats` which will be run when Gitea
start or midnight. It will correct the numbers.
fixes#30565
When using git with a gitea hosted repository, the HTTP-Transport did
honor the user and repository redirects, which are created when renaming
a user or repo and also when transferring ownership of a repo to a
different organization. This is extremely helpful, as repo URLs remain
stable and do not have to be migrated on each client's worktree and
other places, e.g. CI at once.
The SSH transport - which I favor - did not know of these redirections
and I implemented a lookup during the `serv` command.
* repository deletion, delete ephemeral runners with active tasks as
well skips regular cleanup
* user deletion, delete ephemeral runners with active tasks as well
skips regular cleanup
* delete ephemeral runners once status changes to done
* You no longer see used ephemeral runners after the task is done
* if you see one the cron job takes care of it
* fixes a fixture status to upload confirmed
* add another fixture as noise to break tests as soon they are exposed
to api
* v4 delete test added check that artifact is no longer visible in
internal api with status pending delete
* removal of http 404 on empty list: actions/upload-artifact@v4 now
backoff on http 404 of ListArtifacts endpoint
* fixes artifacts with pending delete etc. are able to be found and
downloaded if the storage is not freed
Implements runner apis based on
https://docs.github.com/en/rest/actions/self-hosted-runners?apiVersion=2022-11-28#list-self-hosted-runners-for-an-organization
- Add Post endpoints for registration-token, google/go-github revealed
this as problem
- We should deprecate Get Endpoints, leaving them for compatibility
- Get endpoint of admin has api path /admin/runners/registration-token
that feels wrong, /admin/actions/runners/registration-token seems more
consistent with user/org/repo api
- Get Runner Api
- List Runner Api
- Delete Runner Api
- Tests admin / user / org / repo level endpoints
Related to #33750 (implements point 1 and 2)
Via needs discovered in #32461, this runner api is needed to allow
cleanup of runners that are deallocated without user interaction.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Noticed a SQL in gitea.com has a bigger load. It seems both `is_pull`
and `pin_order` are not indexed columns in the database.
```SQL
SELECT `id`, `repo_id`, `index`, `poster_id`, `original_author`, `original_author_id`, `name`, `content`, `content_version`, `milestone_id`, `priority`, `is_closed`, `is_pull`, `num_comments`, `ref`, `pin_order`, `deadline_unix`, `created_unix`, `updated_unix`, `closed_unix`, `is_locked`, `time_estimate` FROM `issue` WHERE (repo_id =?) AND (is_pull = 0) AND (pin_order > 0) ORDER BY pin_order
```
I came across a comment
https://github.com/go-gitea/gitea/pull/24406#issuecomment-1527747296
from @delvh , which presents a more reasonable approach. Based on this,
this PR will migrate all issue and pull request pin data from the
`issue` table to the `issue_pin` table. This change benefits larger
Gitea instances by improving scalability and performance.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* download endpoint has to use 302 redirect
* fake blob download used if direct download not possible
* downloading v3 artifacts not possible
New repo apis based on GitHub Rest V3
- GET /runs/{run}/artifacts (Cannot use run index of url due to not
being unique)
- GET /artifacts
- GET + DELETE /artifacts/{artifact_id}
- GET /artifacts/{artifact_id}/zip
- (GET /artifacts/{artifact_id}/zip/raw this is a workaround for a http
302 assertion in actions/toolkit)
- api docs removed this is protected by a signed url like the internal
artifacts api and no longer usable with any token or swagger
- returns http 401 if the signature is invalid
- or change the artifact id
- or expired after 1 hour
Closes#33353Closes#32124
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Currently, anyone with write permissions to a repo are able to rename
default or protected branches.
This change follows
[GitHub's](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-branches-in-your-repository/renaming-a-branch)
design by only allowing repo/site admins to change these branches.
However, it also follows are current design for protected branches and
only allows admins to modify branch names == branch protection rule
names. Glob-based rules cannot be renamed by anyone (as was already the
case, but we now catch `ErrBranchIsProtected` which we previously did
not catch, throwing a 500).
This PR fixes inconsistencies between system and default webhooks in the
Gitea API. (See also #26418)
- A system webhook is a webhook that captures events for all
repositories.
- A default webhook is copied to a new repository when it is created.
Before this PR `POST /api/v1/admin/hooks/` creates default webhooks (if
not configured otherwise) and `GET /api/v1/admin/hooks/` returns system
webhooks.
The PR introduces an optional query parameter to `GET
/api/v1/admin/hooks/` to enable selecting if either default, system or
both kind of webhooks should be retrieved. By default the flag is set to
return system webhooks keep current behaviour.
## Examples
### System Webhooks
#### Create
```
POST /api/v1/admin/hooks/
{
"type": "gitea",
"active": false,
"branch_filter": "*",
"events": [ "create", "..." ],
"config": {
"url": "http://...",
"content_type": "json",
"secret": "secret",
"is_system_webhook": true // <-- controls hook type
}
}
```
#### List
```
GET/api/v1/admin/hooks?type=system //type argument is optional here since it's the default
```
#### Others
The other relevant endpoints work as expected by referencing the hook by
id
```
GET /api/v1/admin/hooks/:id
PATCH /api/v1/admin/hooks/:id
DELETE /api/v1/admin/hooks/:id
```
### Default Webhooks
#### Create
```
POST /api/v1/admin/hooks/
{
"type": "gitea",
"active": false,
"branch_filter": "*",
"events": [ "create", "..." ],
"config": {
"url": "http://...",
"content_type": "json",
"secret": "secret",
"is_system_webhook": false // optional, as false is the default value
}
}
```
#### List
```
GET/api/v1/admin/hooks?type=default
```
#### Others
The other relevant endpoints work as expected by referencing the hook by
id
```
GET /api/v1/admin/hooks/:id
PATCH /api/v1/admin/hooks/:id
DELETE /api/v1/admin/hooks/:id
```
Fix#32795
If a job uses a matrix, multiple `ActionRunJobs` may have the same
`JobID`. We need to merge the outputs of these jobs to make them
available to the jobs that need them.
Resolve#31492
The response time for the Pull Requests API has improved significantly,
dropping from over `2000ms` to about `350ms` on my local machine. It's
about `6` times faster.
A key area for further optimization lies in batch-fetching data for
`apiPullRequest.ChangedFiles, apiPullRequest.Additions, and
apiPullRequest.Deletions`.
Tests `TestAPIViewPulls` does exist and new tests added.
- This PR also fixes some bugs in `GetDiff` functions.
- This PR also fixes data inconsistent in test data. For a pull request,
the head branch's reference should be equal to the reference in
`pull/xxx/head`.
When running e2e tests on flaky networks, gravatar can cause a timeout
and test failures. Turn off, and populate avatars on e2e test suite run
to make them reliable.
We have some actions that leverage the Gitea API that began receiving
401 errors, with a message that the user was not found. These actions
use the `ACTIONS_RUNTIME_TOKEN` env var in the actions job to
authenticate with the Gitea API. The format of this env var in actions
jobs changed with go-gitea/gitea/pull/28885 to be a JWT (with a
corresponding update to `act_runner`) Since it was a JWT, the OAuth
parsing logic attempted to parse it as an OAuth token, and would return
user not found, instead of falling back to look up the running task and
assigning it to the actions user.
Make ACTIONS_RUNTIME_TOKEN in action runners could be used,
attempting to parse Oauth JWTs. The code to parse potential old
`ACTION_RUNTIME_TOKEN` was kept in case someone is running an older
version of act_runner that doesn't support the Actions JWT.
In profiling integration tests, I found a couple places where per-test
overhead could be reduced:
* Avoiding disk IO by synchronizing instead of deleting & copying test
Git repository data. This saves ~100ms per test on my machine
* When flushing queues in `PrintCurrentTest`, invoke `FlushWithContext`
in a parallel.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
currently restricted users can only see the repos of teams in orgs they
are part at.
they also should only see the users that are also part at the same team.
---
*Sponsored by Kithara Software GmbH*
Gitea instance keeps reporting a lot of errors like "LFS SSH transfer connection denied, pure SSH protocol is disabled". When starting debugging the problem, there are more problems found. Try to address most of them:
* avoid unnecessary server side error logs (change `fail()` to not log them)
* figure out the broken tests/user2/lfs.git (added comments)
* avoid `migratePushMirrors` failure when a repository doesn't exist (ignore them)
* avoid "Authorization" (internal&lfs) header conflicts, remove the tricky "swapAuth" and use "X-Gitea-Internal-Auth"
* make internal token comparing constant time (it wasn't a serous problem because in a real world it's nearly impossible to timing-attack the token, but good to fix and backport)
* avoid duplicate routers (introduce AddOwnerRepoGitLFSRoutes)
* avoid "internal (private)" routes using session/web context (they should use private context)
* fix incorrect "path" usages (use "filepath")
* fix incorrect mocked route point handling (need to check func nil correctly)
* split some tests from "git general tests" to "git misc tests" (to keep "git_general_test.go" simple)
Still no correct result for Git LFS SSH tests. So the code is kept there
(`tests/integration/git_lfs_ssh_test.go`) and a FIXME explains the details.
Closes https://github.com/go-gitea/gitea/issues/30296
- Adds a DB fixture for actions artifacts
- Adds artifacts test files
- Clears artifacts test files between each run
- Note: I initially initialized the artifacts only for artifacts tests,
but because the files are small it only takes ~8ms, so I changed it to
always run in test setup for simplicity
- Fix some otherwise flaky tests by making them not depend on previous
tests
This is a large and complex PR, so let me explain in detail its changes.
First, I had to create new index mappings for Bleve and ElasticSerach as
the current ones do not support search by filename. This requires Gitea
to recreate the code search indexes (I do not know if this is a breaking
change, but I feel it deserves a heads-up).
I've used [this
approach](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/analysis-pathhierarchy-tokenizer.html)
to model the filename index. It allows us to efficiently search for both
the full path and the name of a file. Bleve, however, does not support
this out-of-box, so I had to code a brand new [token
filter](https://blevesearch.com/docs/Token-Filters/) to generate the
search terms.
I also did an overhaul in the `indexer_test.go` file. It now asserts the
order of the expected results (this is important since matches based on
the name of a file are more relevant than those based on its content).
I've added new test scenarios that deal with searching by filename. They
use a new repo included in the Gitea fixture.
The screenshot below depicts how Gitea shows the search results. It
shows results based on content in the same way as the current version
does. In matches based on the filename, the first seven lines of the
file contents are shown (BTW, this is how GitHub does it).
Resolves#32096
---------
Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
Misspell 0.5.0 supports passing a csv file to extend the list of
misspellings, so I added some common ones from the codebase. There is at
least one typo in a API response so we need to decided whether to revert
that and then likely remove the dict entry.
On creation of an empty project (no template) a default board will be
created instead of falling back to the uneditable pseudo-board.
Every project now has to have exactly one default boards. As a
consequence, you cannot unset a board as default, instead you have to
set another board as default. Existing projects will be modified using a
cron job, additionally this check will run every midnight by default.
Deleting the default board is not allowed, you have to set another board
as default to do it.
Fixes#29873Fixes#14679 along the way
Fixes#29853
Co-authored-by: delvh <dev.lh@web.de>
Refactor the webhook logic, to have the type-dependent processing happen
only in one place.
---
## Current webhook flow
1. An event happens
2. It is pre-processed (depending on the webhook type) and its body is
added to a task queue
3. When the task is processed, some more logic (depending on the webhook
type as well) is applied to make an HTTP request
This means that webhook-type dependant logic is needed in step 2 and 3.
This is cumbersome and brittle to maintain.
Updated webhook flow with this PR:
1. An event happens
2. It is stored as-is and added to a task queue
3. When the task is processed, the event is processed (depending on the
webhook type) to make an HTTP request
So the only webhook-type dependent logic happens in one place (step 3)
which should be much more robust.
## Consequences of the refactor
- the raw event must be stored in the hooktask (until now, the
pre-processed body was stored)
- to ensure that previous hooktasks are correctly sent, a
`payload_version` is added (version 1: the body has already been
pre-process / version 2: the body is the raw event)
So future webhook additions will only have to deal with creating an
http.Request based on the raw event (no need to adjust the code in
multiple places, like currently).
Moreover since this processing happens when fetching from the task
queue, it ensures that the queuing of new events (upon a `git push` for
instance) does not get slowed down by a slow webhook.
As a concrete example, the PR #19307 for custom webhooks, should be
substantially smaller:
- no need to change `services/webhook/deliver.go`
- minimal change in `services/webhook/webhook.go` (add the new webhook
to the map)
- no need to change all the individual webhook files (since with this
refactor the `*webhook_model.Webhook` is provided as argument)
Fix for regressions introduced by #28805
Enabled projects on repos created before the PR weren't detected. Also,
the way projects mode was detected in settings didn't match the way it
was detected on permission check, which leads to confusion.
Co-authored-by: Giteabot <teabot@gitea.io>
Part of #23318
Add menu in repo settings to allow for repo admin to decide not just if
projects are enabled or disabled per repo, but also which kind of
projects (repo-level/owner-level) are enabled. If repo projects
disabled, don't show the projects tab.
---------
Co-authored-by: delvh <dev.lh@web.de>
Zettat123
Lunny Xiao
koalajoe23
wxiaoguang
ChristopherHX
bytedream
NorthRealm
hiifong
TheFox0x7
Kemal Zebari
Michael B.
Rowan Bohde
6543
Kyle D.
Bruno Sofiato
yp05327
KN4CK3R
silverwind
Denys Konovalov
oliverpool