Commit Graph

53 Commits

Author SHA1 Message Date
Harshavardhana a50ea92c64
feat: introduce list_quorum="auto" to prefer quorum drives (#18084)
NOTE: This feature is not retro-active; it will not cater to previous transactions
on existing setups. 

To enable this feature, please set ` _MINIO_DRIVE_QUORUM=on` environment
variable as part of systemd service or k8s configmap. 

Once this has been enabled, you need to also set `list_quorum`. 

```
~ mc admin config set alias/ api list_quorum=auto` 
```

A new debugging tool is available to check for any missing counters.
2023-12-29 15:52:41 -08:00
Harshavardhana 3c2e1a87e2
fix: support dropping privileges with arbitrary users (#18386)
fixes #18380
2023-11-03 14:18:18 -07:00
Harshavardhana 8c4561b8da
add all missing go.mod for debugging tools (#18049) 2023-09-18 13:47:03 -07:00
Harshavardhana 28a01f0320
update missing license header in files (#17603) 2023-07-08 10:42:05 -07:00
ferhat elmas 3423028713
cleanup Go linter settings (#16736) 2023-03-04 20:57:35 -08:00
Anis Elleuch 7e73fc2870
Implement inspect data API v2 (#15474)
Co-authored-by: Klaus Post <klauspost@gmail.com>
2022-11-02 13:36:38 -07:00
Harshavardhana d76c508566
debug: verify diff on latest objects on source and target buckets (#15069) 2022-06-10 16:56:51 -07:00
Kaan Kabalak e2dd3e3587
Include the entirety of vendor folder in .gitignore (#15046)
The 'go mod vendor' command generates a directory called 
'vendor' in the main module's root directory, which includes 
the required packages to support builds. Therefore, we can 
include the 'vendor' directory in .gitignore completely, 
regardless of any file extension.
2022-06-06 20:47:51 -07:00
Harshavardhana 7e3a7d7044
add healing for invalid shards by skipping the blocks (#13978)
Built on top of #13945, now we need to simply skip the
shards and its automated.
2021-12-23 23:01:46 -08:00
Harshavardhana 0e3037631f
skip inconsistent shards if possible (#13945)
data shards were wrong due to a healing bug
reported in #13803 mainly with unaligned object
sizes.

This PR is an attempt to automatically avoid
these shards, with available information about
the `xl.meta` and actually disk mtime.
2021-12-21 10:08:26 -08:00
Harshavardhana 4fa250a6a1
add hash-set debugging currently supports SIPMOD (#13911)
fixes a bug in s3-check-md5 for single part
uploaded multipart objects.
2021-12-15 12:07:15 -08:00
Harshavardhana 92fdcafb66
add verification tests for ETag on replicated content (#13857) 2021-12-07 10:08:26 -08:00
Harshavardhana 41d4d650e4
simplify dockerfiles and remove duplication (#12419)
also add additional packages such as 'iproute',
'iputils' for `ping` and `ip addr` commands.
2021-06-03 08:16:06 -07:00
Harshavardhana d825d92499 rename production to release directory, rebuild assets 2021-04-25 16:51:29 -07:00
Nitish Tiwari e5fb6294a7 Remove healthcheck script for Docker image (#8095)
There are multiple possibilities for running MinIO within
a container e.g. configurable address, non-root user etc.
This makes it difficult to identify actual IP / Port to
use to check healthcheck status from within a container.

It is simpler to use external healthcheck mechanisms
like healthcheck command in docker-compose to check
for MinIO health status. This is similar to how checks
work in Kubernetes as well.

This PR removes the healthcheck script used inside
Docker container and ad documentation on how to
use docker-compose based healthcheck mechanism.
2019-08-17 12:44:04 -07:00
Harshavardhana 55dd017e62 Deprecate auto detection of container user (#7930)
There is no reliable way to handle fallbacks for
MinIO deployments, due to various command line
options and multiple locations which require
access inside container.

Parsing command line options is tricky to figure
out which is the backend disk etc, we did try
to fix this in implementations of check-user.go
but it wasn't complete and introduced more bugs.

This PR simplifies the entire approach to rather
than running Docker container as non-root by default
always, it allows users to opt-in. Such that they
are aware that that is what they are planning to do.

In-fact there are other ways docker containers can
be run as regular users, without modifying our
internal behavior and adding more complexities.
2019-07-17 19:20:55 +01:00
Harshavardhana 1dc25bcf5f
Add mint tests into MinIO repo (#7886) 2019-07-09 18:32:39 -07:00
Harshavardhana 91ceae23d0 Add support for customizable user (#7569) 2019-06-10 20:27:42 +05:30
Nitish Tiwari 1e82c4a7c4
Implement Docker healthcheck script in Go (#7105)
Go script makes it easy to read/maintain. Also updated the timeout
in Dockerfiles from 5s to default 30s and test interval to 1m

Higher timeout makes sense as server may sometimes respond slowly
if under high load as reported in #6974

Fixes #6974
2019-02-20 21:42:03 +05:30
Harshavardhana 0e02328c98 Migrate config.json from config-dir to backend (#6195)
This PR is the first set of changes to move the config
to the backend, the changes use the existing `config.json`
allows it to be migrated such that we can save it in on
backend disks.

In future releases, we will slowly migrate out of the
current architecture.

Fixes #6182
2018-08-15 10:11:47 +05:30
poornas a3e806ed61 Add disk based edge caching support. (#5182)
This PR adds disk based edge caching support for minio server.

Cache settings can be configured in config.json to take list of disk drives,
cache expiry in days and file patterns to exclude from cache or via environment
variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY

Design assumes that Atime support is enabled and the list of cache drives is
fixed.
 - Objects are cached on both GET and PUT/POST operations.
 - Expiry is used as hint to evict older entries from cache, or if 80% of cache
   capacity is filled.
 - When object storage backend is down, GET, LIST and HEAD operations fetch
   object seamlessly from cache.

Current Limitations
 - Bucket policies are not cached, so anonymous operations are not supported in
   offline mode.
 - Objects are distributed using deterministic hashing among list of cache
   drives specified.If one or more drives go offline, or cache drive
   configuration is altered - performance could degrade to linear lookup.

Fixes #4026
2018-03-28 14:14:06 -07:00
Harshavardhana b4ae2bd2f5 Remove snap reference from the repo (#5676)
snap builds are not well tested, since
some releases have stopped working. Remove
their reference to keep it simple.

Fixes #5674
Fixes #5472
2018-03-20 09:33:02 +05:30
poornas 25107c2e11 Add NAS gateway support (#5516) 2018-02-20 12:21:12 -08:00
David G f4d4ea5c36 Implement Sia Gateway (#5114) 2017-11-22 12:12:10 -08:00
Andreas Auernhammer ea65350308 add vscode config files to gitignore (#4855) 2017-08-27 13:44:39 -07:00
Leo Arias 617f2394fb Hardcode snap version while a store bug is fixed (#4806)
* Hardcode snap version while a store bug is fixed
This is a workaround for for a failure in the store, which limits the version to 32 characters.
https://forum.snapcraft.io/t/versions-can-be-at-most-32-characters/1642
Capitalize the summary
* add the snapcraft dirs to gitignore bring back settings.json to ignore
2017-08-14 11:12:46 -07:00
Andreas Auernhammer 1e8925cea0 add settings.json to .gitignore (#4786)
settings.json is the VS code workspace settings file
2017-08-08 13:06:57 -07:00
Harshavardhana 064c51162d api: Add new ListenBucketNotificationHandler. (#2336)
This API is precursor before implementing `minio lambda` and `mc` continous replication.

This new api is an extention to BucketNofication APIs.

// Request
```
GET /bucket?notificationARN=arn:minio:lambda:us-east-1:10:minio HTTP/1.1
...
...
```

// Response
```

{"Records": ...}
...
...
...
{"Records": ...}
```
2016-08-04 22:01:58 -07:00
Harshavardhana 90c20a8c11 Add codecov for minio. (#2359) 2016-08-04 16:48:50 -07:00
Harshavardhana d1fa1d9352 Remove binary files from previous commit. 2016-05-10 15:49:17 -07:00
Harshavardhana 379e0abf03 cleanup: Remove old donut/xl code and erasure implementation.
This is a change to bring in 'klauspost/reedsolomon' library
in #1270 patch.
2016-04-02 17:30:35 -07:00
Harshavardhana 408aa72146 build/vet: Fix all the shadowing reports with go1.6
Golang 1.6 is default version for the build now.

Additionally set 'GODEBUG=cgocheck=0' for now, until
we fix the erasure coding package.

Readmore here https://tip.golang.org/doc/go1.6#cgo
2016-02-23 14:34:39 -08:00
Harshavardhana 2181003609 web: Removing dependency for gpg and downloading assets.
Assets are vendorized from now on and updated for each release.
2016-02-23 13:32:12 -08:00
Harshavardhana 62f6ffb6db xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
Harshavardhana a066184bed ui-assets: Integrate UI assets. 2016-02-04 18:07:05 -08:00
Harshavardhana d55f72f09a Add doc change. 2016-02-01 02:45:19 -08:00
Harshavardhana 7845515f36 build: Versioning now populated through ldflags 2015-11-02 02:37:26 -08:00
Frederick F. Kautz IV 172cd18455 Adding initial logging framework to api_router 2015-04-28 20:37:08 -07:00
Anand Babu (AB) Periasamy 3c4aa85c69 git commit hash generator 2015-03-25 22:57:30 -07:00
Harshavardhana 7ce3ab3193 Restructure minio api, move signature checks from utils to Api. 2015-03-18 18:16:19 -07:00
Frederick F. Kautz IV 9da26de696 Adding IDEA files to gitignore 2015-03-11 12:05:12 -07:00
Harshavardhana 3a3c8645fc Update comments across the codebase 2015-03-03 02:39:38 -08:00
Harshavardhana c3ad0906e0 Add deadcode code which recursivley goes into all directories and verifies dangling variables. 2015-03-02 18:44:20 -08:00
Harshavardhana d036bf3a2a Restructure docs and move mkdocs.yml to top-level 2015-02-08 17:40:39 -08:00
Frederick F. Kautz IV 68205c8c6c Adding sublime workspace to gitignore 2015-01-30 17:39:08 -08:00
Frederick F. Kautz IV 82a69d3ab9 Adding initial test suites 2015-01-25 11:22:38 -08:00
Harshavardhana 432275e966 Full restructure in accordance with
- pkg/{subsystem}/{package} style
  - modify Makefile to reflect the new style,
    consolidate various entries
  - add a dummy ``main.go`` at top level
2015-01-14 11:29:04 -08:00
Frederick F. Kautz IV 2e8a68a796 Adding cover.out to gitignore 2014-11-22 18:31:49 -08:00
Harshavardhana 1e7515a7df Add erasure coding and decoding using Intel Storage Acceleration library
- move contrib/erasure --> contrib/isal
 - bring in low level 'isal' package for Go for exposing C functions
 - Implement Erasure 'encoding'
   Supports - Reed Solomon Codes, Cauchy Codes
 - Implement Erasure 'decoding'
   Supports - Reed Solomon Codes, Cauchy Codes
 - Renames Minios -> Minio at all the references
2014-11-13 15:20:18 -08:00
Frederick F. Kautz IV 5878dd5e50 Cleaning up more code, system generated by config 2014-11-06 22:45:27 -08:00