神刀安全网

Scalable reverse image search built on Kubernetes and Elasticsearch

Scalable reverse image search built on Kubernetes and Elasticsearch

Scalable reverse image search

built on Kubernetes and Elasticsearch

Scalable reverse image search built on Kubernetes and Elasticsearch

Pavlov Matchmakes it easy to search for images that look similar to each other. Using a state-of-the-art perceptual hash, it is invariant to scaling and rotation. Its HTTP API is quick to integrate and flexible for a number of reverse image search applications. Kubernetes and Elasticsearch allow Match to scale to billions of images with ease while giving you full control over where your data is stored. Match uses the awesomeascribe/image-match under the hood for most of the image search legwork.

Getting Started

$ docker run -e ELASTICSEARCH_URL=https://daisy.us-west-1.es.amazonaws.com -it pavlov/match 

Match is packaged as a Docker container ( pavlov/match on Docker Hub), making it highly portable and scalable to billions of images. You can configure a few options using environment variables:

  • ELASTICSEARCH_URL (required)

    A URL pointing to the Elasticsearch database where image signatures are to be stored. If you don’t want to host your own Elasticsearch cluster, consider using AWS Elasticsearch Service . That’s what we use.

  • ELASTICSEARCH_INDEX (default: images)

    The index in the Elasticsearch database where image signatures are to be stored.

  • ELASTICSEARCH_DOC_TYPE (default: images)

    The doc type used for storing image signatures.

  • WORKER_COUNT (default: 4)

    The number of gunicorn worker forks to maintain in each Docker container.

One-command deployment with spread

Match is particularly awesomesauce when integrated into the Kubernetes container orchestration architecture. spread makes it easy to get Match up and running quickly:

$ go get rsprd.com/spread/cmd/spread $ git clone https://github.com/pavlovml/match $ vim .k2e/secret.yml # configure me $ spread deploy . 

Using in a custom Kubernetes cluster

You can configure the service, replication controller, and secret like so:

# match-service.yml apiVersion: v1 kind: Service metadata:   namespace: default   name: match spec:   ports:   - name: http     port: 80     protocol: TCP   selector:     app: match
# match-rc.yml apiVersion: v1 kind: ReplicationController metadata:   namespace: default   name: match spec:   replicas: 2   selector:     app: match   template:     metadata:       labels:         app: match     spec:       containers:       - name: match         image: pavlov/match:latest         ports:         - containerPort: 80         env:         - name: WORKER_COUNT           valueFrom:             secretKeyRef:               name: match               key: worker-count         - name: ELASTICSEARCH_URL           valueFrom:             secretKeyRef:               name: match               key: elasticsearch.url         - name: ELASTICSEARCH_INDEX           valueFrom:             secretKeyRef:               name: match               key: elasticsearch.index         - name: ELASTICSEARCH_DOC_TYPE           valueFrom:             secretKeyRef:               name: match               key: elasticsearch.doc-type
# match-secret.yml apiVersion: v1 kind: Secret metadata:   namespace: default   name: match data:   # 4, base64 encoded   worker-count: NA==    # https://daisy.us-west-1.es.amazonaws.com (change me)   elasticsearch.url: aHR0cHM6Ly9kYWlzeS51cy13ZXN0LTEuZXMuYW1hem9uYXdzLmNvbQ==    # images   elasticsearch.index: aW1hZ2Vz    # images   elasticsearch.doc-type: aW1hZ2Vz

API

Match has a simple HTTP API. All request parameters are specified via application/x-www-form-urlencoded or multipart/form-data .

POST /add

Adds an image signature to the database.

Parameters

  • urlor image (required)

    The image to add to the database. It may be provided as a URL via url or as a multipart/form-data file upload via image .

  • filepath (required)

    The path to save the image to in the database. If another image already exists at the given path, it will be overwritten.

Example Response

{   "status": "ok",   "error": [],   "method": "add",   "result": [] }

DELETE /delete

Deletes an image signature from the database.

Parameters

  • filepath (required)

    The path of the image signature in the database.

Example Response

{   "status": "ok",   "error": [],   "method": "delete",   "result": [] }

POST /search

Searches for a similar image in the database. Scores range from 0 to 100, with 100 being a perfect match.

Parameters

  • urlor image (required)

    The image to add to the database. It may be provided as a URL via url or as a multipart/form-data file upload via image .

Example Response

{   "status": "ok",   "error": [],   "method": "search",   "result": [     {       "score": 99.0,       "filepath": "http://static.wixstatic.com/media/0149b5_345c8f862e914a80bcfcc98fcd432e97.jpg_srz_614_709_85_22_0.50_1.20_0.00_jpg_srz"     }   ] }

POST /compare

Compares two images, returning a score for their similarity. Scores range from 0 to 100, with 100 being a perfect match.

Parameters

  • url1or image1 , url2 or image2 (required)

    The images to compare. They may be provided as a URL via url1 / url2 or as a multipart/form-data file upload via image1 / image2 .

Example Response

{   "status": "ok",   "error": [],   "method": "compare",   "result": [     {       "score": 99.0     }   ] }

GET /count

Count the number of image signatures in the database.

Example Response

{   "status": "ok",   "error": [],   "method": "list",   "result": [420] }

GET /list

Lists the file paths for the image signatures in the database.

Parameters

  • offset (default: 0)

    The location in the database to begin listing image paths.

  • limit (default: 20)

    The number of image paths to retrieve.

Example Response

{   "status": "ok",   "error": [],   "method": "list",   "result": [     "http://img.youtube.com/vi/iqPqylKy-bY/0.jpg",     "https://i.ytimg.com/vi/zbjIwBggt2k/hqdefault.jpg",     "https://s-media-cache-ak0.pinimg.com/736x/3d/67/6d/3d676d3f7f3031c9fd91c10b17d56afe.jpg"   ] }

GET /ping

Check for the health of the server.

Example Response

{   "status": "ok",   "error": [],   "method": "ping",   "result": [] }

Development

$ export ELASTICSEARCH_URL=https://daisy.us-west-1.es.amazonaws.com $ make build $ make run $ make push 

License and Acknowledgements

Match is based onascribe/image-match, which is in turn based on the paper An image signature for any kind of image , Goldberg et al . There is an existing reference implementation which may be more suited to your needs.

Match itself is released under theBSD 3-Clause license. ascribe/image-match is released under the Apache 2.0 license.

转载本站任何文章请注明:转载至神刀安全网,谢谢神刀安全网 » Scalable reverse image search built on Kubernetes and Elasticsearch

分享到:更多 ()

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
分享按钮