Card forensics using VLM localized prompt approach

#11
by vijayksagi - opened

Hi

we have passports of different countries for which we need to check card forensics on below points:
Ex: UAE passport
passport

  1. check background pattern of the UAE passport is tampered
  2. Face is present on the left hand side of the UAE passport
  3. Logo is present on top right corner of the UAE passport

we need localized model by which we can train using our images. pls suggest

The approach we are following is using for point (1) we need to extract background extract feature vector and extracted prompt for the pattern
and store in json file. Later on for every image compare with json can classify as tampered or original
Ex:
{
"pattern_vector": [... 768 numbers ...],
"prompt_respone": "Diagonal guilloche mesh, light-gray watermark..."
}

using VIT encoder extract the pattern vector
using LLM prompt we extract the prompt response

Guys pls suggest if some better idea

Thanks
vijay

vijayksagi changed discussion title from Card forensics using VLM prompt and images for training the model to Card forensics using VLM prompt for forensics
vijayksagi changed discussion title from Card forensics using VLM prompt for forensics to Card forensics using VLM localized prompt approach
Blog-explorers org

This might be a better question for the huggingface forum... You won't get answers here.

https://discuss.huggingface.co/

@ccocks-deca thanks for the quick support

Sign up or log in to comment