This is a raw beta so expect lots of things to change and improve over time.
To install mirror run
pip install git+https://github.com/FrancescoSaverioZuppichini/mirror.git
Basic example:
from mirror import mirror
from mirror.visualisations import *
from PIL import Image
from torchvision.models import resnet101, resnet18, vgg16, alexnet
from torchvision.transforms import ToTensor, Resize, Compose
# create a model
model = vgg16(pretrained=True)
# get an image
cat = Image.open("./cat.jpg")
# resize the image and make it a tensor
input = Compose([Resize((224,224)), ToTensor()])(cat)
# add 1 dim for batch
input = input.unsqueeze(0)
# call mirror with the input and the model
mirror(input, model, visualisations=[DeepDreamVis, BackPropVis, GradCamVis])It will automatic open a new tab in your browser
By clicking on the radio button 'guide', all the relus negative output will be set to zero producing a nicer looking image
- Add text field for class
-
You can find an example below
from mirror.visualisations.Visualisation import Visualisation
class DummyVisualisation(Visualisation):
def __call__(self, inputs, layer):
return inputs.repeat(self.params['repeat']['value'],1, 1, 1)
@property
def name(self):
return 'dummy'
def init_params(self):
return {'repeat' : {
'type' : 'slider',
'min' : 1,
'max' : 100,
'value' : 3,
'step': 1,
'params': {}
}} The __call__ function is called each time you click a layer or change a value in the options on the right.
The init_params function returns a dictionary of options that will be showed on the right drawer of the application. For now only slider and radio are supported
- Cache reused layer
- Make a generic abstraction of a visualisation in order to add more features
- Add dropdown as parameter
- Add text field
- Support multiple inputs
- Support multiple models
- Add all visualisation present here https://github.com/utkuozbulak/pytorch-cnn-visualizations
- Gradient visualization with vanilla backpropagation
- Gradient visualization with guided backpropagation [1]
- Gradient visualization with saliency maps [4]
- Gradient-weighted [3] class activation mapping [2]
- Guided, gradient-weighted class activation mapping [3]
- Smooth grad [8]
- CNN filter visualization [9]
- Inverted image representations [5]
- Deep dream [10]
- Class specific image generation [4]
- Grad Cam