Monthly Archives: February 2014

Find Similar and Duplicate PSD- Photoshop Document Files (Adobe Photoshop) – Duplicate Photo Finder

Duplicate Photo Finder

Find Similar and Duplicate PSD Files

How to find similar or duplicate photo files – PSD- Photoshop Document Files (Adobe Photoshop)

Photoshop files can grow quite large and saving multiple versions of the same document can lead to huge waste of disk space. Having in mind that even simpler files can take up to 100 MB, a quantity of even 50 unused files will waste 5 GB of disk space. Visual Similarity Duplicate Image Finder provides support for over 100 popular image formats and also 300 RAW camera formats. This of course includes the popular *.psd file format.

While a standard duplicate finder can find exact duplicates, it will not help you in such situation where the images are slightly modified or converted to multiple formats. The Duplicate Photo Finder performs a real image analysis and can identify duplicate and similar photos even if they are edited and stored in various file formats. It is the only similarity tool that supports the complex PSD file format. There is a FREE demo of this low-cost tool available for download at the above link.

About Adobe Photoshop

Adobe Photoshop is a graphics editing program developed and published by Adobe Systems.

Adobe’s 2003 “Creative Suite” rebranding led to Adobe Photoshop 8’s renaming to Adobe Photoshop CS. Thus, Adobe Photoshop CS6 is the 13th major release of Adobe Photoshop. The CS rebranding also resulted in Adobe offering numerous software packages containing multiple Adobe programs for a reduced price. Adobe Photoshop is released in two editions: Adobe Photoshop, and Adobe Photoshop Extended, with the Extended having extra 3D image creation, motion graphics editing, and advanced image analysis features.[7] Adobe Photoshop Extended is included in all of Adobe’s Creative Suite offerings except Design Standard, which includes the Adobe Photoshop edition. However, in 2013, Adobe announced a new brand – Creative Cloud. And the Photoshop version included in this suite is named Photoshop CC. The CC version( Originally 14.0, and will be continuously maintained and upgraded) has 3D content as Photoshop Extended, which means the standard version is canceled.

Alongside Photoshop and Photoshop Extended, Adobe also publishes Photoshop Elements and Photoshop Lightroom, collectively called “The Adobe Photoshop Family”. In 2008, Adobe released Adobe Photoshop Express, a free web-based image editing tool to edit photos directly on blogs and social networking sites. In 2011, a version was released for the Android operating system and the iOS operating system, followed by a release of a version for Windows 8 in 2013.[8][9][10]

Adobe distributes Photoshop in Windows and Macintosh versions.

Free Download: Duplicate Photo Finder

PSD File format

Photoshop files have default file extension as .PSD, which stands for “Photoshop Document.” A PSD file stores an image with support for most imaging options available in Photoshop. These include layers with masks, transparency, text, alpha channels and spot colors, clipping paths, and duotone settings. This is in contrast to many other file formats (e.g. .JPG or .GIF) that restrict content to provide streamlined, predictable functionality. A PSD file has a maximum height and width of 30,000 pixels, and a length limit of 3 Gigabytes.
Photoshop files sometimes have the file extension .PSB, which stands for “Photoshop Big” (also known as “large document format”). A PSB file extends the PSD file format, increasing the maximum height and width to 300,000 pixels and the length limit to around 4 Exabytes. The dimension limit was apparently chosen arbitrarily by Adobe, not based on computer arithmetic constraints (it is not close to a power of two, as is 30,000) but for ease of software testing. PSD and PSB formats are documented.[15]
Because of Photoshop’s popularity, PSD files are widely used and supported to some extent by most competing software. The .PSD file format can be exported to and from Adobe’s other apps like Adobe Illustrator, Adobe Premiere Pro, and After Effects, to make professional standard DVDs and provide non-linear editing and special effects services, such as backgrounds, textures, and so on, for television, film, and the web. Photoshop’s primary strength is as a pixel-based image editor, unlike vector-based image editors. Photoshop also enables vector graphics editing through its Paths, Pen tools, Shape tools, Shape Layers, Type tools, Import command, and Smart Object functions. These tools and commands are convenient to combine pixel-based and vector-based images in one Photoshop document, because it may not be necessary to use more than one program. To create very complex vector graphics with numerous shapes and colors, it may be easier to use software that was created primarily for that purpose, such as Adobe Illustrator or CorelDRAW. Photoshop’s non-destructive Smart Objects can also import complex vector shapes.[16]

Photoshop Language availabilities

Photoshop is available in more than 20 languages. These are the following:

Chinese Simplified, Chinese Traditional, Czech, Danish, Dutch, Finnish, French, German, Hungarian, Italian, Japanese, Korean, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, Romanian, Turkish and Ukrainian. Arabic and Hebrew are added as of Photoshop CS6.[17]

Free Download: Duplicate Photo Finder

Photoshop Plugins

Photoshop functionality can be extended by add-on programs called Photoshop plugins (or plug-ins). Adobe creates some plugins, such as Adobe Camera Raw, but third-party companies develop most plugins, according to Adobe’s specifications. Some are free and some are commercial software. Most plugins work with only Photoshop or Photoshop-compatible hosts, but a few can also be run as standalone applications.

There are various types of plugins, such as filter, export, import, selection, color correction, and automation. The most popular plugins are the filter plugins (also known as a 8bf plugins), available under the Filter menu in Photoshop. Filter plugins can either modify the current image or create content. Below are some popular types of plugins, and some well-known companies associated with them:

Color correction plugins (Alien Skin Software,[18] Nik Software,[19] OnOne Software,[20] Topaz Labs Software,[21] The Plugin Site,[22] etc.)
Special effects plugins (Alien Skin Software, Auto FX Software,[23] AV Bros.,[24] Flaming Pear Software,[25] etc.)
3D effects plugins (Andromeda Software,[26] Strata,[27] etc.)

Adobe Camera Raw (also known as ACR and Camera Raw) is a special plugin, supplied free by Adobe, used primarily to read and process raw image files so that the resultant images can be processed by Photoshop.[28] It can also be used from within Adobe Bridge.

Free Download: Duplicate Photo Finder

How to Find Similar JPG Files (JPEG – Joint Photographic Experts Group) – Find Duplicate JPG Files

Find Duplicate Photos

Duplicate Photo Finder

How to find duplicate or similar photo files – .jpg, .jpeg, .jpe, .jif, .jfif, .jfi

Visual Similarity Duplicate Image Finder support over 100 popular image formats and also 300 RAW camera formats. The various JPEG file formats are just  a small part of the long list of supported image formats. The tool can find exact duplicates regardless of the image format and also similar images that vary in color, crop size, watermarks and so on. Download a FREE Duplicate Photo Finder demo.

A description of the JPEG file format follows below.

In computing, JPEG (/ˈdʒeɪpɛɡ/ JAY-peg)[1] (seen most often with the .jpg extension) is a commonly used method of lossy compression for digital images, particularly for those images produced by digital photography. The degree of compression can be adjusted, allowing a selectable tradeoff between storage size and image quality. JPEG typically achieves 10:1 compression with little perceptible loss in image quality.

JPEG compression is used in a number of image file formats. JPEG/Exif is the most common image format used by digital cameras and other photographic image capture devices; along with JPEG/JFIF, it is the most common format for storing and transmitting photographic images on the World Wide Web.[citation needed] These format variations are often not distinguished, and are simply called JPEG.

The term “JPEG” is an acronym for the Joint Photographic Experts Group, which created the standard. The MIME media type for JPEG is image/jpeg (defined in RFC 1341), except in Internet Explorer, which provides a MIME type of image/pjpeg when uploading JPEG images.[2]

JPEG/JFIF supports a maximum image size of 65535×65535.[3]

The JPEG standard

The name “JPEG” stands for Joint Photographic Experts Group, the name of the committee that created the JPEG standard and also other still picture coding standards. The “Joint” stood for ISO TC97 WG8 and CCITT SGVIII. In 1987 ISO TC 97 became ISO/IEC JTC1 and in 1992 CCITT became ITU-T. Currently on the JTC1 side JPEG is one of two sub-groups of ISO/IEC Joint Technical Committee 1, Subcommittee 29, Working Group 1 (ISO/IEC JTC 1/SC 29/WG 1) – titled as Coding of still pictures.[4][5][6] On the ITU-T side ITU-T SG16 is the respective body. The original JPEG group was organized in 1986,[7] issuing the first JPEG standard in 1992, which was approved in September 1992 as ITU-T Recommendation T.81[8] and in 1994 as ISO/IEC 10918-1.

The JPEG standard specifies the codec, which defines how an image is compressed into a stream of bytes and decompressed back into an image, but not the file format used to contain that stream.[9] The Exif and JFIF standards define the commonly used file formats for interchange of JPEG-compressed images.

JPEG standards are formally named as Information technology – Digital compression and coding of continuous-tone still images. ISO/IEC 10918 consists of the following parts:

Typical usage

The JPEG compression algorithm is at its best on photographs and paintings of realistic scenes with smooth variations of tone and color. For web usage, where the amount of data used for an image is important, JPEG is very popular. JPEG/Exif is also the most common format saved by digital cameras.

On the other hand, JPEG may not be as well suited for line drawings and other textual or iconic graphics, where the sharp contrasts between adjacent pixels can cause noticeable artifacts. Such images may be better saved in a lossless graphics format such as TIFF, GIF, PNG, or a raw image format. The JPEG standard actually includes a lossless coding mode, but that mode is not supported in most products.

As the typical use of JPEG is a lossy compression method, which somewhat reduces the image fidelity, it should not be used in scenarios where the exact reproduction of the data is required (such as some scientific and medical imaging applications and certain technical image processing work).

JPEG is also not well suited to files that will undergo multiple edits, as some image quality will usually be lost each time the image is decompressed and recompressed, particularly if the image is cropped or shifted, or if encoding parameters are changed – see digital generation loss for details. To avoid this, an image that is being modified or may be modified in the future can be saved in a lossless format, with a copy exported as JPEG for distribution.

JPEG compression

JPEG uses a lossy form of compression based on the discrete cosine transform (DCT). This mathematical operation converts each frame/field of the video source from the spatial (2D) domain into the frequency domain (aka transform domain.) A perceptual model based loosely on the human psychovisual system discards high-frequency information, i.e. sharp transitions in intensity, and color hue. In the transform domain, the process of reducing information is called quantization. In simpler terms, quantization is a method for optimally reducing a large number scale (with different occurrences of each number) into a smaller one, and the transform-domain is a convenient representation of the image because the high-frequency coefficients, which contribute less to the over picture than other coefficients, are characteristically small-values with high compressibility. The quantized coefficients are then sequenced and losslessly packed into the output bitstream. Nearly all software implementations of JPEG permit user control over the compression-ratio (as well as other optional parameters), allowing the user to trade off picture-quality for smaller file size. In embedded applications (such as miniDV, which uses a similar DCT-compression scheme), the parameters are pre-selected and fixed for the application.

The compression method is usually lossy, meaning that some original image information is lost and cannot be restored, possibly affecting image quality. There is an optional lossless mode defined in the JPEG standard. However, this mode is not widely supported in products.

There is also an interlaced “Progressive JPEG” format, in which data is compressed in multiple passes of progressively higher detail. This is ideal for large images that will be displayed while downloading over a slow connection, allowing a reasonable preview after receiving only a portion of the data. However, support for progressive JPEGs is not universal. When progressive JPEGs are received by programs that do not support them (such as versions of Internet Explorer before Windows 7)[13] the software only displays the image after it has been completely downloaded.

There are also many medical imaging and traffic systems that create and process 12-bit JPEG images, normally grayscale images. The 12-bit JPEG format has been part of the JPEG specification for some time, but this format is not as widely supported.

Syntax and structure

A JPEG image consists of a sequence of segments, each beginning with a marker, each of which begins with a 0xFF byte followed by a byte indicating what kind of marker it is. Some markers consist of just those two bytes; others are followed by two bytes indicating the length of marker-specific payload data that follows. (The length includes the two bytes for the length, but not the two bytes for the marker.) Some markers are followed by entropy-coded data; the length of such a marker does not include the entropy-coded data. Note that consecutive 0xFF bytes are used as fill bytes for padding purposes, although this fill byte padding should only ever take place for markers immediately following entropy-coded scan data (see JPEG specification section B.1.1.2 and E.1.2 for details; specifically “In all cases where markers are appended after the compressed data, optional 0xFF fill bytes may precede the marker”).

Within the entropy-coded data, after any 0xFF byte, a 0x00 byte is inserted by the encoder before the next byte, so that there does not appear to be a marker where none is intended, preventing framing errors. Decoders must skip this 0x00 byte. This technique, called byte stuffing (see JPEG specification section F.1.2.3), is only applied to the entropy-coded data, not to marker payload data. Note however that entropy-coded data has a few markers of its own; specifically the Reset markers (0xD0 through 0xD7), which are used to isolate independent chunks of entropy-coded data to allow parallel decoding, and encoders are free to insert these Reset markers at regular intervals (although not all encoders do this).

Lossless editing

See also: jpegtran and Commons:User:Cropbot

A number of alterations to a JPEG image can be performed losslessly (that is, without recompression and the associated quality loss) as long as the image size is a multiple of 1 MCU block (Minimum Coded Unit) (usually 16 pixels in both directions, for 4:2:0 chroma subsampling). Utilities that implement this include jpegtran, with user interface Jpegcrop, and the JPG_TRANSFORM plugin to IrfanView.

Blocks can be rotated in 90 degree increments, flipped in the horizontal, vertical and diagonal axes and moved about in the image. Not all blocks from the original image need to be used in the modified one.

The top and left edge of a JPEG image must lie on a 8 × 8 pixel block boundary, but the bottom and right edge need not do so. This limits the possible lossless crop operations, and also prevents flips and rotations of an image whose bottom or right edge does not lie on a block boundary for all channels (because the edge would end up on top or left, where – as aforementioned – a block boundary is obligatory).

When using lossless cropping, if the bottom or right side of the crop region is not on a block boundary then the rest of the data from the partially used blocks will still be present in the cropped file and can be recovered. It is also possible to transform between baseline and progressive formats without any loss of quality, since the only difference is the order in which the coefficients are placed in the file.

Furthermore, several JPEG images can be losslessly joined together, as long as the edges coincide with block boundaries.

JPEG files

The file format known as “JPEG Interchange Format” (JIF) is specified in Annex B of the standard. However, this “pure” file format is rarely used, primarily because of the difficulty of programming encoders and decoders that fully implement all aspects of the standard and because of certain shortcomings of the standard:

Color space definition
Component sub-sampling registration
Pixel aspect ratio definition.

Several additional standards have evolved to address these issues. The first of these, released in 1992, was JPEG File Interchange Format (or JFIF), followed in recent years by Exchangeable image file format (Exif) and ICC color profiles. Both of these formats use the actual JIF byte layout, consisting of different markers, but in addition employ one of the JIF standard’s extension points, namely the application markers: JFIF use APP0, while Exif use APP1. Within these segments of the file, that were left for future use in the JIF standard and aren’t read by it, these standards add specific metadata.

Thus, in some ways JFIF is a cutdown version of the JIF standard in that it specifies certain constraints (such as not allowing all the different encoding modes), while in other ways it is an extension of JIF due to the added metadata. The documentation for the original JFIF standard states:[14]

JPEG File Interchange Format is a minimal file format which enables JPEG bitstreams to be exchanged between a wide variety of platforms and applications. This minimal format does not include any of the advanced features found in the TIFF JPEG specification or any application specific file format. Nor should it, for the only purpose of this simplified format is to allow the exchange of JPEG compressed images.

Image files that employ JPEG compression are commonly called “JPEG files”, and are stored in variants of the JIF image format. Most image capture devices (such as digital cameras) that output JPEG are actually creating files in the Exif format, the format that the camera industry has standardized on for metadata interchange. On the other hand, since the Exif standard does not allow color profiles, most image editing software stores JPEG in JFIF format, and also include the APP1 segment from the Exif file to include the metadata in an almost-compliant way; the JFIF standard is interpreted somewhat flexibly.[15]

Strictly speaking, the JFIF and Exif standards are incompatible because they each specify that their marker segment (APP0 or APP1, respectively) appears first. In practice, most JPEG files contain a JFIF marker segment that precedes the Exif header. This allows older readers to correctly handle the older format JFIF segment, while newer readers also decode the following Exif segment, being less strict about requiring it to appear first.

JPEG filename extensions

The most common filename extensions for files employing JPEG compression are .jpg and .jpeg, though .jpe, .jfif and .jif are also used. It is also possible for JPEG data to be embedded in other file types – TIFF encoded files often embed a JPEG image as a thumbnail of the main image; and MP3 files can contain a JPEG of cover art, in the ID3v2 tag.

Color profile

Many JPEG files embed an ICC color profile (color space). Commonly used color profiles include sRGB and Adobe RGB. Because these color spaces use a non-linear transformation, the dynamic range of an 8-bit JPEG file is about 11 stops; see gamma curve.

Effects of JPEG compression

JPEG compression artifacts blend well into photographs with detailed non-uniform textures, allowing higher compression ratios. Notice how a higher compression ratio first affects the high-frequency textures in the upper-left corner of the image, and how the contrasting lines become more fuzzy. The very high compression ratio severely affects the quality of the image, although the overall colors and image form are still recognizable. However, the precision of colors suffer less (for a human eye) than the precision of contours (based on luminance). This justifies the fact that images should be first transformed in a color model separating the luminance from the chromatic information, before subsampling the chromatic planes (which may also use lower quality quantization) in order to preserve the precision of the luminance plane with more information bits.

Lossless further compression

From 2004 to 2008, new research has emerged on ways to further compress the data contained in JPEG images without modifying the represented image.[22][23][24][25] This has applications in scenarios where the original image is only available in JPEG format, and its size needs to be reduced for archival or transmission. Standard general-purpose compression tools cannot significantly compress JPEG files.

Typically, such schemes take advantage of improvements to the naive scheme for coding DCT coefficients, which fails to take into account:

Correlations between magnitudes of adjacent coefficients in the same block;
Correlations between magnitudes of the same coefficient in adjacent blocks;
Correlations between magnitudes of the same coefficient/block in different channels;
The DC coefficients when taken together resemble a downscale version of the original image multiplied by a scaling factor. Well-known schemes for lossless coding of continuous-tone images can be applied, achieving somewhat better compression than the Huffman coded DPCM used in JPEG.

Some standard but rarely used options already exist in JPEG to improve the efficiency of coding DCT coefficients: the arithmetic coding option, and the progressive coding option (which produces lower bitrates because values for each coefficient are coded independently, and each coefficient has a significantly different distribution). Modern methods have improved on these techniques by reordering coefficients to group coefficients of larger magnitude together;[22] using adjacent coefficients and blocks to predict new coefficient values;[24] dividing blocks or coefficients up among a small number of independently coded models based on their statistics and adjacent values;[23][24] and most recently, by decoding blocks, predicting subsequent blocks in the spatial domain, and then encoding these to generate predictions for DCT coefficients.[25]

Typically, such methods can compress existing JPEG files between 15 and 25 percent, and for JPEGs compressed at low-quality settings, can produce improvements of up to 65%.[24][25]

A freely available tool called packJPG[26] is based on the 2007 paper “Improved Redundancy Reduction for JPEG Files.”

VSDIF – Professional Duplicate Photo Finder Adds Support For Sidecar Files (Adobe Photoshop, LightRoom XMP etc.)

Find Similar Images

Duplicate Photo Finder

 

VSDIF 5.2.0.1 Adds Support For Adobe Photoshop XMP and Other File Format Sidecar Files

This release provides an option in the settings where sidecar file extensions can be listed. Multiple sidecar extensions like XMP, THM, info, Exif etc. can be listed simultaneously using a vertical line as separator – “|”. The move/delete operations over the selected files will also affect the corresponding sidecar files that are set in the settings.

This release fixes also the Pause/Resume functionality where some issues were detected. This includes also the missing icon. An issue with the exclude and auto-check states on project reload was identified and resolved.

Some additional changes are applied to the cache system. The active cache name is stored and reused when new projects are created. If there is no cache, a DEFAULT cache will be created and used automatically. Every project file stores its own cache settings that will be reloaded upon project reload.

Save project and shutdown/log off/etc. are now available in the DEMO version.

What is a Sidecar File?

Sidecar files, also known as buddy files or connected files, are files which store data (often metadata) which is not supported by the source file format.

For each source file one or more sidecar files can be created. This is in contrast to “metadata databases” where one database contains metadata for several source files.

In most cases the relationship between the source file and the sidecar file is based on the file name; sidecar files have the same base name as the source file, but with a different extension. The problem with this system is that most operating systems and file managers have no knowledge of these relationships, and might allow the user to rename or move one of the files thereby breaking the relationship.

Download Duplicate Photo Finder Here

Do you need a feature that we have missed to implement? Please comment below or e-mail us. We will be glad to hear from you!