l'image de fond

fr       en

Optical Flow

Block based

Method

This method consist to find moves in imgage for each blocks the image. For a block size of M*N, we divide the image into block with the next way:

To observe the movement ongoing between two images, we use blocks of the second image as a reference block. For each reference block, we study the moving of itself with the block corresponding of the first image and its 8 neighbour blocks. For the previous diagram, if the block studied is the number 5 of the image at t we search moves for blocks from number 1 to 9 of the image at t+1. We can modelised blocks of the algorythm by the next model

The aim is to use the M*N block of the image at t+1 as a mask in the window of the image at t. We can performed 4 computations:

Mean square erreur

M S E ( i , j ) = 1 M N ∑ m = 1 M ∑ n = 1 N ( X m , n - Y m + i , n + j ) 2

Mean absolut erreur

M A D ( i , j ) = 1 M N ∑ m = 1 M ∑ n = 1 N | X m , n - Y m + i , n + j |

sum absolut difference

S A D ( i , j ) = ∑ m = 1 M ∑ n = 1 N | X m , n - Y m + i , n + j |

cross correlation function

C C F ( i , j ) = ∑ m = 1 M ∑ n = 1 N X m , n Y m + i , n + j [ ∑ m = 1 M ∑ n = 1 N { X m , n 2 } ] 1 2 [ ∑ m = 1 M ∑ n = 1 N Y m + i , n + j 2 ] 1 2

Where X is the block of the image at t+1 and Y the window of the image at t, i and j are pixels of the window. Attention to the size limitations !!!.

Now, for each results of each block we have two different cases:

Program

clear all;
close all;


[filename, pathname] = uigetfile({'*.jpg;*.tif;*.png;*.gif;*.bmp','All Image Files';...
          '*.*','All Files' },'mytitle',...
          'C:\Work\myfile.jpg');

x = imread(filename);
x=double(x);
M1=x;
M2=x;
M2(100:150,100:150)=x(103:153,103:153);

[l c]=size(M1);
n=32;  
M = floor(l/n);
N = floor(c/n);
for i = 1 : n-2
    for j = 1 : n-2
        blockref=M1( i*M+1 : (i+1)*M , j*N+1 : (j+1)*N);
        blockcurrent=M2( (i-1)*M+1 : ((i+1)+1)*M , (j-1)*N+1 : ((j+1)+1)*N);
        [tx ty]=size(blockcurrent);
        for x = 1 : tx - M
            for y = 1 : ty - N
               MSE(x,y)= sum(sum((blockref - blockcurrent(x:x+(M-1),y:y+(N-1))).^2))/(M*N);
               MAD(x,y)=sum(sum(abs(blockref - blockcurrent(x:x+(M-1),y:y+(N-1)))))/(M*N);
               SAD(x,y)=sum(sum(abs(blockref - blockcurrent(x:x+(M-1),y:y+(N-1)))));
               cross1 = sum(sum( blockref.*blockcurrent(x:x+(M-1),y:y+(N-1)) ));
               cross2 = sum(sum( blockref.^2 ))^0.5;
               cross3 = sum(sum( blockcurrent(x:x+(M-1),y:y+(N-1)).^2 ))^0.5;
               CCF(x,y)= cross1/(cross2*cross3);
               
            end
        end        
       
%MSE MAD et SADvminimisation
       if MSE(M,N)==0
           vmse(i,j,1)= 0;
           vmse(i,j,2)= 0;
       else
        [valx minx] = min(SAD);
        [valy miny] = min(valx);
        minx=minx(miny);
        vmse(i,j,1)=minx-(M+1);
        vmse(i,j,2)=miny-(N+1);
       end
        if MAD(M,N)==0
           vmad(i,j,1)= 0;
           vmad(i,j,2)= 0;
       else
        [valx minx] = min(SAD);
        [valy miny] = min(valx);
        minx=minx(miny);
        vmad(i,j,1)=minx-(M+1);
        vmad(i,j,2)=miny-(N+1);
        end
        if SAD(M,N)==0
           vsad(i,j,1)= 0;
           vsad(i,j,2)= 0;
       else
        [valx minx] = min(SAD);
        [valy miny] = min(valx);
        minx=minx(miny);
        vsad(i,j,1)=minx-(M+1);
        vsad(i,j,2)=miny-(N+1);
        end

%CCF maximisation
       if CCF(M,N)== max(max(CCF))
           vccf(i,j,1)= 0;
           vccf(i,j,2)= 0;
       else
        [valx maxx] = max(CCF);
        [valy maxy] = max(valx);
        maxx=maxx(maxy);
        vccf(i,j,1)=maxx-M;
        vccf(i,j,2)=maxy-N;
       end
    end
end
 
figure(  'Name','Optical Flow: Block Based',...
            'NumberTitle','off',...
            'color',[0.3137 0.3137 0.5098]);
subplot(221)
imshow(uint8(M2))
hold on
quiver([M*2 : M : (n-1)*M]'*ones(1,n-2),ones(1,n-2)'*[N*2 : N : (n-1)*N],vmse(:,:,1),vmse(:,:,2),'r')
title('Mean Square Error')
subplot(222)
imshow(uint8(M2))
hold on
quiver([M*2 : M : (n-1)*M]'*ones(1,n-2),ones(1,n-2)'*[N*2 : N : (n-1)*N],vmad(:,:,1),vmad(:,:,2),'r')
title('Mean Absolute Difference')
subplot(223)
imshow(uint8(M2))
hold on
quiver([M*2 : M : (n-1)*M]'*ones(1,n-2),ones(1,n-2)'*[N*2 : N : (n-1)*N],vsad(:,:,1),vsad(:,:,2),'r')
title('Sum  Absolute Difference')
subplot(224)
imshow(uint8(M2))
hold on
quiver([M*2 : M : (n-1)*M]'*ones(1,n-2),ones(1,n-2)'*[N*2 : N : (n-1)*N],vccf(:,:,1),vccf(:,:,2),'r')
title('Cross Correlation function')

Examples





Problem

Those algorythms of optical flow detect only the translating moves. If moves are rotating or Zoom they will be misdetect.

Copyright © 2010-2014, all rights reserved, contact: operationpixel@free.fr