As outlined in the package home page, GrANd is about applying the GAN functionalities on "more structured pictures" such as floor plans or maps for instance. Now, this model is about the application in architecture and interior design. If you are someone interested in this model with a background in these professions, and would like to suggest requirements, improvements or collaborations, please feel free to leave a reply below, or send me a message.
https://maicro879342585.wordpress.com/2021/08/04/grand-gan-with-a-grid/
Model Structure
I'm abstracting out a floor plan into arrays of sqft "pixels", while allowing for various sizes of mixtures. This number of mixtures is analogous to the 3 (red, blue, green) in a normal picture. I intend them to work like layers or features, but there are too many "layers" and "features" in computer vision, so I'm using the term mixture, although it has nothing to do with a Gaussian mixture. We can look at each mixture as having a different focus. Examples of focuses can be the likes of bedroom proportion or water supply points, but possibly more abstract issues such as the feeling of openness.
When each sqft has a quantitative measure in each of the properties or focuses, the model is essentially working on the spatial arrangement of things. My hope is that we can use GAN to learn the spatial patterns in various designs for functionality or style. The success of this algorithm working on pictures gave me confidence. Yet, I am still very curious to see what kind of "pictures" a GAN would actually "draw" in this case, and any rational or irrational results it would generate.
I also think there should be an important part of the procedure where the resulting spatial arrangements generated by the GAN get "interpreted" into a more formal interior design type of drawing. Functionalities such as identifying where the walls should be, fact checking for client-specific requirements, etc. may well require quite a bit of machine intelligence on their own. Still, I don't imagine loads of AI for this part. The idea is to let the GAN go a little wild if it needs to, while afterwards we build in some controls for fitting to the applications.
Moving Pieces
Now, let's talk about the actual code files.
https://github.com/nancy123lin/GAN-with-a-Grid/blob/main/floorPlan.py
<Comment below to give me suggestions to have you credited, build on my work and credit me, or have me build a similar model for you.>
the FloorPlan object
I am using object oriented coding for the more application-related stuff. In floorPlan.py, I've created a FloorPlan class. Its get_grid() function creates the sqft pixel "space" for the floor plan, which is ready to be fed into a GAN.
As mentioned above, another functionality that should be included under the floor_plan class is to convert a grid floor plan into a common format floor plan. As discussed later in the "reading floor plan data" subsection, this part might vary depending on the format of output required. However, certain capabilities would be needed anyways, and that is the ability to "fuzzy match" the grid output with a formatted floor plan. When adjacent sqft's are a bedroom and a bathroom, it is pretty clear that there should be a wall right at the boundary; but if you are weighing in a heat map for the feeling of openness, the choice of putting a wall or not, or using a piece of clear glass or a wooden grid may not be so automatic. What's needed is a set of procedures in accordance with our model setup and design logic.
using GANs
Then, there is the GAN itself of course. I will start with an existing model. We can choose to go as deep into the source code as we want. As a baseline, we can simply manage to feed our data with a mixture size of 3 into the equivalent red, blue and green channels of the GAN model. Now, that brings us to the next important component of our model: input mapping.
reading floor plan data
Reading floor plan files is very straight forward as humans speak, but would involve quite a bit of coding when it is being realized. We need to translate the floor plan files into a FloorPlan object or into a certain format compatible with the FloorPlan object. As far as Maicro knows, there are several widely used data formats and markup languages in the industry, such as DXF or DWG for CAD drawings, or the likes of Shapefile and IndoorGML(https://www.researchgate.net/publication/316871250_Modeling_Standards_and_File_Formats_for_Indoor_Mapping" target="_blank"). If you would like to provide insight or data on some architectural files, or collaborate in certain file formats, leave me a reply below or read more on collaborations (coming soon). For now, I'll make it a computer vision exercise to read from pictures.
As a computer vision module, we could first recognize the straight and curvy lines, text and numbers, as well as objects such as a sink, a window, a sofa, etc. We can then determine the lines to be walls and doors, and put the objects into categories of interior structures. Text labelling in floor plans are usually quite succinct and standardized, so I would expect some existing text recognition algorithms to be more than sufficient for our problem. Putting the walls, objects and text together, we can then finalize what each sqft is. More specifically, we can determine a value for every sqft with respect to what each element of the mixture stands for according to the application definition.
Further Work
I will be setting up floor plan reading and grid to floor plan functionalities, as well as connecting our model to a GAN. If we consider potential added capabilities down the road, we may look into a 3D model. As always, if you have any ideas or suggestions, feel free to comment below or contact me.