Vector database sharding
Multimodal search
JaguarDB quantization
JaguarDB Vector API
Best Vector databases
JaguarDB in Docker
Setup JaguarDB with tar package
Setup JaguarDB on multiple nodes
Vector index sharing
How zeromove works
Video introduction
|
|
Example: Multimodal Search
Multi-Modal architectures leverage more than one domain to learn a specific task. CLIP combines Natural Language Processing and Computer Vision. JaguarDB empowers users to discover similar images not only through traditional image inputs but also via textual descriptions, embracing a versatile and multimodal approach. This method allows for data retrieval based on the semantic meanings conveyed through both textual and visual signals, enhancing the flexibility and comprehensiveness of the search process.
The following Python example demonstrates extracting embeddings from text and searching relevant images. You will need "pip install -U jaguardb-http-client" to have the jaguar http client package.
from PIL import Image
from sentence_transformers import SentenceTransformer
from jaguardb_http_client.JaguarHttpClient import JaguarHttpClient
url="http://192.168.8.88:8080/fwww/"
jag = JaguarHttpClient(url)
apikey = "demouser"
token = jag.login( apikey )
q= "create store imgvec (v vector(512, 'euclidean_fraction_float'), v:img file, cat char(16))"
jag.get(q, token)
model = SentenceTransformer('clip-ViT-B-32')
images = [ Image.open('img1.jpg') ]
embeddings = model.encode( images )
comma_sep_str = ",".join( [str(x) for x in embeddings[0] ])
jag.postFile(token, 'imag1.jpg', 2 )
q = "insert into imgvec values ( '" + comma_sep_str + "', 'img1.jpg', 'outdoor')"
jag.post(q, token, True)
# add more images here ...
# prepare a query text
qs = "give me some images of parking lot"
texts = [ qs ]
embeddings = model.encode( texts )
comma_sep_str = ",".join( [str(x) for x in embeddings[0] ])
qs = "select similarity(v, '" + embeddings + "','type=euclidean_fraction_float,topk=3') from imgvec"
resp = jag.post(qs, token)
jarr = json.loads(resp.text)
while rec in jarr:
print(json.loads(rec))
The program should output some images that contain parking lot scenes, if the database has stored such images.
|
