convert log1.txt
compile log.txt
@bogdannedelcu There seems to be a problem when I converting to edgeTPU.
The converted model you uploaded is boxed normally.

I assume image size must not be higher than 224, as all Google samples for EdgeTPU are like this

The EgdeTPU supports arbitrarily large inputs, provided they fit into memory. Eg 1024px is fine if the model is small enough.

@jveitchmichaelis looking at yolov5s I see it starts with an image size of 640, so all the internal weights are much larger than if it started at 224, say 4x times larger. As I tested previously even for the 224 input resolution as input value, the compiled model does not get fully compiled on the EdgeTPU probably because the TRANSPOSE operation has too many parameters (as seen in @ksp7518 "compile log.txt" file)

My suggestion is to create a "Yolo5xs" version which starts at a lower resolution and maybe fewer inner layers as to fit completely in EdgeTPU. The 224 value I took from the fact that all Google samples for Coral are built using this value, so basically this could be the upper bound resolution value we could fit inside this processor. Just a supposition though... since I understand Google somehow created the EfficientNets-EdgeTPU by brute force search given the parameters of the EdgeTPU.

See here most of the models benchmarked are at 224: https://coral.ai/docs/edgetpu/benchmarks/

Given the Pixel 6 phone includes the EdgeTPU there is a good motivation to have a YoloV5 version which fits inside the EdgeTPU

I got the same issue, can anyone lead us to the solution?

Hello @AmeerAlahmadi, if you start with a model like YOLOv5n6u fine-tuned with default parameters of Ultralytics hub like this one: https://hub.ultralytics.com/models/VNKIIDQ0ngjulGQyQnSg?tab=train

You may export the model directly from the web interface of Ultralytics with a reasonable 256x256 input size which performs all operations in memory. You may see in this image as the entire model is exported as one operation on the EdgeTPU

Please note how it performs in this type of scenario where I finetuned it on VisDrone dataset: https://youtu.be/cdaIjJBYFMo . Bear in mind performance is not great for small objects as image resolution is low. However I think there could be strategies to mitigate this.

@jveitchmichaelis repo is a great place to start because hi is eliminating torch dependency: https://github.com/jveitchmichaelis/edgetpu-yolo . I recently added some specific code to implement the new output format of the last layer of the Yolov5n6u model which changed since 2020 (the 1x14x1360 last tensor). Please see here: https://github.com/bogdannedelcu/edgetpu-yolo in ms.py and edgetpumodel.py.

By the way I intent to write a paper on this, mainly related to drone usage of Google Coral & Yolo5-8, if anybody is interested to join let me know.

Hello. I want to try this but I'm using 'Coral USB Accelerator' on the PC.
This support to 'TPU' but the 'Coral USB Accelerator' need 'tflite-runtime' and 'edgeTPU'.
Is this possible to running this on my PC?

@ksp7518 We will be pushing out a new update in the next few days that should support both tflite-runtime and edgetpu (we are prioritizing tflite runtime). We had some internal issues so our development got delayed so sorry about that. If you want, I can tag you whenever we release the update.

@shivamswarnkar I hope to test new version. Thank you for reply

@batrlatom @glenn-jocher please find my submission here: https://github.com/jveitchmichaelis/edgetpu-yolo

Notes:

• Minimal dependency solution - no Torch required
• Tested on a Jetson Nano, but should work on the Coral too (again, minimal deps). It will probably also work on a PC (@ksp7518)
• Docker container provided with a minimal dev environment
• Extensive installation notes for hardware, software and export
• Speed benchmarks for speed are provided for 96/192/224 px inputs; also some notes on possibilities for larger models.
• I've included an option to benchmark on COCO Val2017 - it will output a prediction json similar to val.py. There is also a script to process the json using pycocotools, but I don't consider this a deployment dep. mAP results are in the readme. Not great, but somewhat expected for a post-quantised model running at very low resolution.

All compiled models with image size > 96 pixels have one TRANSPOSE operation running on CPU probably due to size constraints. Therefore all the subsequent operations run on CPU, which is why you have the 100% usage rate. See above the error "Operation is otherwise supported, but not mapped due to some unspecified limitation" (TRANSPOSE and QUANTIZE)"

@bogdannedelcu Thank you for reply. I have another question. Is there a performance difference between 'coco128' and 'coco' in the conversion script?
!python3 models/tf.py --weights yolov5m.pt --cfg models/yolov5m.yaml --tfl-int8 --no-tfl-detect --img 320 --tf-raw-resize --source data/coco128.yaml --ncalib 200
my USB accelerator is possible to running --img 320 after comfile with yolov5m.pt
but the detection performance is not good and I have wonder about the option coco128.yaml
and I had test in colab but the storage was full. so, I'm testing in the local.

The source of the calibration images is not super important. I quantised my models by pointing the script at the val2017 folder and using ~1k samples.

How are you exporting the model? With the latest version of the EdgeTPU compiler, I wasn't able to compile beyond 224 px - I also tried using delegate search which didn't help. I didn't try with the m model actually. I guess it's because you're not using the Detection layer. In that case I was also able to export up to around 320 px with yolov5s, however speed was not particularly good.

Can you post the model here?

@jveitchmichaelis
Hello, here is my model
How to check detection layer?
yolov5m-int8_edgetpu_320_coco128.zip

@bogdannedelcu Hello, I tested the image size with 96. The comfile log was good ,all operation maped to Edgetpu.
but the CPU usage was same 100%.
I wonder what is the problem.
--------------------------------------------------------------------------------------------------------------------------

@jveitchmichaelis thank you for reply
I test about CPU usage. If it is set to 'cpu' when using edgetpu, isn't there a problem?

<detect.py>
img = torch.zeros((1, 3, *imgsz), device=device) # init img

The cpu seems to go to 100% when accessing the 'img' variable in the loop that processing stream image of frame
How to set device value in case of edgetpu?

You can't set the device to be EdgeTPU - it doesn't work with Torch. What the script probably does is load the data, convert it to Torch and then to Numpy for the TFlite interpreter. This is partly why I dropped Torch in my implementation, because it's unnecessary (you might as well go straight to TFLite). However I don't have a pure numpy implementation of the detection layer currently.

Can you paste the output from the profiler?

Have a look at my repository for an example - at least see if you can run it with the models I've converted (they're smaller, but should work). If that works, check to see how your CPU usage is doing. I'm curious if you see a similar load.

Note that OpenCV will often use 100% CPU accessing a camera or something similar, though I understand you're processing a video file?

@jveitchmichaelis Hello, I trying testing. but I got the error. I leaved error message to your github.
Thank you

Thanks Stefani, I've messaged you. Look forward to hearing the results!

Is it working?

@csikosaron98 yes, at least for the config at the time of submission. It's possible things have changed with some of the more recent nano models. Please check out the repository, if you have any questions/issues, please let me know there.

https://github.com/jveitchmichaelis/edgetpu-yolo

@AmeerAlahmadi see my previous answer above #3428 (reply in thread)

I fine-tuned one pretrained model and the architecture changed a little bit, but you will see in my repo it is working quite well. No need to use the old script I posted in 2020. You may export the trained model from the Ultralytics web interface.

I'll try this. thank a lot.